Jul 16 16:49:10.536977 ip-10-0-128-50 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 16 16:49:10.536988 ip-10-0-128-50 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 16 16:49:10.536994 ip-10-0-128-50 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 16 16:49:10.537273 ip-10-0-128-50 systemd[1]: Failed to start Kubernetes Kubelet. Jul 16 16:49:20.553999 ip-10-0-128-50 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 16 16:49:20.554012 ip-10-0-128-50 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 8518e6f7d1b64081840d92aaf5e8bf62 -- Jul 16 16:50:46.972000 ip-10-0-128-50 systemd[1]: Starting Kubernetes Kubelet... Jul 16 16:50:47.538015 ip-10-0-128-50 kubenswrapper[2571]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 16:50:47.538015 ip-10-0-128-50 kubenswrapper[2571]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 16 16:50:47.538015 ip-10-0-128-50 kubenswrapper[2571]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 16:50:47.538015 ip-10-0-128-50 kubenswrapper[2571]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 16 16:50:47.538015 ip-10-0-128-50 kubenswrapper[2571]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 16:50:47.538015 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.418368 2571 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 16 16:50:47.538015 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423455 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 16:50:47.538015 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423466 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 16:50:47.538015 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423469 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 16:50:47.538015 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423474 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 16:50:47.538015 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423477 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 16:50:47.538015 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423480 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 16:50:47.538015 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423483 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 16:50:47.538015 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423486 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 16:50:47.538015 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423489 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 16:50:47.557143 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423492 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 16:50:47.557143 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423495 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 16:50:47.557143 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423497 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 16:50:47.557143 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423500 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 16:50:47.557143 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423503 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 16:50:47.557143 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423505 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 16:50:47.557143 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423508 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 16:50:47.557143 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423510 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 16:50:47.557143 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423512 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 16:50:47.557143 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423515 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 16:50:47.557143 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423517 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 16:50:47.557143 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423520 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 16:50:47.557143 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423522 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 16:50:47.557143 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423528 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 16:50:47.557143 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423531 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 16:50:47.557143 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423534 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 16:50:47.557143 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423537 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 16:50:47.557143 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423541 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 16:50:47.557143 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423545 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 16:50:47.498397 ip-10-0-128-50 systemd[1]: Started Kubernetes Kubelet. Jul 16 16:50:47.573695 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423548 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 16:50:47.573695 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423551 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 16:50:47.573695 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423553 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 16:50:47.573695 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423556 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 16:50:47.573695 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423559 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 16 16:50:47.573695 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423561 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 16:50:47.573695 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423564 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 16:50:47.573695 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423567 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 16:50:47.573695 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423570 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 16:50:47.573695 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423572 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 16:50:47.573695 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423575 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 16:50:47.573695 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423577 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 16:50:47.573695 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423580 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 16:50:47.573695 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423583 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 16:50:47.573695 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423585 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 16:50:47.573695 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423588 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 16:50:47.573695 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423590 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 16:50:47.573695 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423593 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 16:50:47.573695 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423596 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 16:50:47.573695 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423599 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 16:50:47.576679 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423602 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 16:50:47.576679 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423606 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 16:50:47.576679 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423608 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 16:50:47.576679 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423611 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 16:50:47.576679 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423615 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 16:50:47.576679 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423617 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 16:50:47.576679 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423621 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 16:50:47.576679 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423623 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 16:50:47.576679 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423626 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 16:50:47.576679 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423628 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 16:50:47.576679 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423631 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 16:50:47.576679 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423633 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 16:50:47.576679 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423636 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 16:50:47.576679 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423638 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 16:50:47.576679 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423641 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 16:50:47.576679 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423643 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 16:50:47.576679 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423646 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 16:50:47.576679 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423649 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 16:50:47.576679 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423651 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 16:50:47.726468 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423654 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 16:50:47.726468 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423657 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 16:50:47.726468 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423659 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 16:50:47.726468 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423662 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 16:50:47.726468 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423664 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 16:50:47.726468 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423667 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 16:50:47.726468 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423669 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 16:50:47.726468 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423672 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 16:50:47.726468 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423674 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 16:50:47.726468 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423676 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 16:50:47.726468 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423679 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 16:50:47.726468 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423681 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 16:50:47.726468 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423685 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 16:50:47.726468 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423689 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 16:50:47.726468 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423692 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 16:50:47.726468 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423694 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 16:50:47.726468 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423697 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 16:50:47.726468 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423701 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 16:50:47.726468 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423703 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 16:50:47.727151 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423706 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 16:50:47.727151 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423715 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 16:50:47.727151 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423717 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 16:50:47.727151 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423720 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 16:50:47.727151 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423723 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 16:50:47.727151 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423725 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 16:50:47.727151 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423727 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 16:50:47.727151 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423730 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 16:50:47.727151 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423733 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 16:50:47.727151 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423747 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 16:50:47.727151 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423749 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 16:50:47.727151 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423752 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 16:50:47.727151 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.423755 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 16:50:47.727151 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424439 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 16:50:47.727151 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424445 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 16:50:47.727151 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424448 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 16:50:47.727151 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424451 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 16:50:47.727151 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424454 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 16:50:47.727151 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424459 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 16:50:47.727151 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424462 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 16:50:47.728023 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424465 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 16:50:47.728023 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424467 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 16:50:47.728023 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424470 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 16:50:47.728023 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424472 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 16:50:47.728023 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424475 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 16:50:47.728023 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424477 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 16:50:47.728023 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424480 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 16 16:50:47.728023 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424482 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 16:50:47.728023 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424485 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 16:50:47.728023 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424487 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 16:50:47.728023 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424490 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 16:50:47.728023 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424493 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 16:50:47.728023 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424495 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 16:50:47.728023 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424498 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 16:50:47.728023 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424500 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 16:50:47.728023 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424503 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 16:50:47.728023 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424505 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 16:50:47.728023 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424508 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 16:50:47.728023 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424510 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 16:50:47.728023 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424513 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 16:50:47.728673 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424515 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 16:50:47.728673 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424517 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 16:50:47.728673 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424520 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 16:50:47.728673 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424522 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 16:50:47.728673 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424525 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 16:50:47.728673 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424527 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 16:50:47.728673 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424530 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 16:50:47.728673 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424535 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 16:50:47.728673 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424538 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 16:50:47.728673 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424542 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 16:50:47.728673 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424546 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 16:50:47.728673 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424552 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 16:50:47.728673 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424555 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 16:50:47.728673 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424558 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 16:50:47.728673 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424561 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 16:50:47.728673 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424563 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 16:50:47.728673 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424565 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 16:50:47.728673 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424568 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 16:50:47.728673 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424570 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 16:50:47.728673 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424573 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 16:50:47.729522 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424575 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 16:50:47.729522 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424578 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 16:50:47.729522 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424580 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 16:50:47.729522 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424582 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 16:50:47.729522 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424585 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 16:50:47.729522 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424587 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 16:50:47.729522 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424590 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 16:50:47.729522 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424592 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 16:50:47.729522 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424595 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 16:50:47.729522 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424597 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 16:50:47.729522 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424600 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 16:50:47.729522 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424602 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 16:50:47.729522 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424604 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 16:50:47.729522 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424607 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 16:50:47.729522 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424609 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 16:50:47.729522 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424612 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 16:50:47.729522 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424614 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 16:50:47.729522 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424616 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 16:50:47.729522 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424620 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 16:50:47.729522 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424624 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 16:50:47.730145 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424626 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 16:50:47.730145 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424629 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 16:50:47.730145 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424633 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 16:50:47.730145 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424635 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 16:50:47.730145 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424645 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 16:50:47.730145 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424648 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 16:50:47.730145 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424651 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 16:50:47.730145 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424653 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 16:50:47.730145 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424655 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 16:50:47.730145 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424658 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 16:50:47.730145 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424663 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 16:50:47.730145 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424665 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 16:50:47.730145 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424668 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 16:50:47.730145 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424670 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 16:50:47.730145 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424673 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 16:50:47.730145 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424675 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 16:50:47.730145 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424679 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 16:50:47.730145 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424682 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 16:50:47.730145 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424684 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 16:50:47.730145 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424687 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 16:50:47.730988 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424689 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 16:50:47.730988 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424692 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 16:50:47.730988 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424694 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 16:50:47.730988 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424697 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 16:50:47.730988 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424699 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 16:50:47.730988 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424702 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 16:50:47.730988 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424704 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 16:50:47.730988 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424706 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 16:50:47.730988 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424709 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 16:50:47.730988 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424712 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 16:50:47.730988 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424714 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 16:50:47.730988 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.424718 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 16:50:47.730988 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425060 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 16:50:47.730988 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425066 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 16:50:47.730988 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425070 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 16:50:47.730988 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425073 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 16:50:47.730988 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425077 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 16:50:47.730988 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425080 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 16:50:47.730988 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425084 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 16:50:47.731614 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425086 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 16:50:47.731614 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425089 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 16:50:47.731614 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425091 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 16:50:47.731614 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425098 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 16:50:47.731614 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425100 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 16:50:47.731614 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425103 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 16:50:47.731614 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425105 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 16:50:47.731614 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425108 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 16:50:47.731614 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425110 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 16:50:47.731614 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425113 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 16:50:47.731614 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425115 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 16:50:47.731614 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425118 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 16:50:47.731614 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425123 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 16:50:47.731614 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425126 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 16:50:47.731614 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425128 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 16:50:47.731614 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425131 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 16:50:47.731614 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425133 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 16:50:47.731614 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425136 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 16:50:47.731614 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425138 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 16:50:47.731614 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425141 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 16:50:47.732663 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425144 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 16:50:47.732663 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425146 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 16:50:47.732663 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425149 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 16:50:47.732663 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425151 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 16:50:47.732663 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425154 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 16:50:47.732663 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425156 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 16:50:47.732663 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425158 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 16:50:47.732663 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425162 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 16:50:47.732663 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425165 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 16:50:47.732663 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425167 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 16:50:47.732663 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425170 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 16:50:47.732663 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425173 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 16:50:47.732663 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425175 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 16:50:47.732663 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425178 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 16:50:47.732663 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425182 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 16:50:47.732663 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425186 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 16:50:47.732663 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425190 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 16:50:47.732663 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425192 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 16:50:47.732663 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425195 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 16:50:47.733303 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425198 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 16:50:47.733303 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425202 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 16:50:47.733303 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425205 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 16:50:47.733303 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425207 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 16:50:47.733303 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425210 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 16:50:47.733303 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425212 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 16:50:47.733303 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425215 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 16:50:47.733303 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425217 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 16:50:47.733303 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425220 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 16:50:47.733303 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425222 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 16:50:47.733303 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425225 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 16:50:47.733303 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425227 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 16:50:47.733303 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425229 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 16:50:47.733303 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425232 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 16:50:47.733303 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425234 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 16:50:47.733303 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425237 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 16:50:47.733303 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425239 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 16:50:47.733303 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425242 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 16:50:47.733303 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425244 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 16:50:47.733303 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425247 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 16:50:47.733956 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425252 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 16:50:47.733956 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425256 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 16:50:47.733956 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425259 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 16:50:47.733956 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425262 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 16 16:50:47.733956 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425265 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 16:50:47.733956 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425268 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 16:50:47.733956 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425271 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 16:50:47.733956 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425274 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 16:50:47.733956 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425278 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 16:50:47.733956 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425280 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 16:50:47.733956 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425283 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 16:50:47.733956 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425285 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 16:50:47.733956 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425288 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 16:50:47.733956 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425290 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 16:50:47.733956 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425293 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 16:50:47.733956 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425295 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 16:50:47.733956 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425298 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 16:50:47.733956 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425300 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 16:50:47.733956 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425303 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 16:50:47.734459 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425305 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 16:50:47.734459 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425307 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 16:50:47.734459 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425310 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 16:50:47.734459 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425312 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 16:50:47.734459 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425315 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 16:50:47.734459 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425317 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 16:50:47.734459 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425319 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 16:50:47.734459 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425322 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 16:50:47.734459 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425324 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 16:50:47.734459 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425327 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 16:50:47.734459 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425329 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 16:50:47.734459 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425332 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 16:50:47.734459 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425334 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 16:50:47.734459 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.425338 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 16:50:47.734459 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425412 2571 flags.go:64] FLAG: --address="0.0.0.0" Jul 16 16:50:47.734459 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425426 2571 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 16 16:50:47.734459 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425436 2571 flags.go:64] FLAG: --anonymous-auth="true" Jul 16 16:50:47.734459 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425442 2571 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 16 16:50:47.734459 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425449 2571 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 16 16:50:47.734459 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425454 2571 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 16 16:50:47.734459 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425458 2571 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 16 16:50:47.735101 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425464 2571 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 16 16:50:47.735101 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425467 2571 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 16 16:50:47.735101 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425470 2571 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 16 16:50:47.735101 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425474 2571 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 16 16:50:47.735101 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425477 2571 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 16 16:50:47.735101 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425480 2571 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 16 16:50:47.735101 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425483 2571 flags.go:64] FLAG: --cgroup-root="" Jul 16 16:50:47.735101 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425486 2571 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 16 16:50:47.735101 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425488 2571 flags.go:64] FLAG: --client-ca-file="" Jul 16 16:50:47.735101 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425491 2571 flags.go:64] FLAG: --cloud-provider="external" Jul 16 16:50:47.735101 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425494 2571 flags.go:64] FLAG: --cluster-dns="[]" Jul 16 16:50:47.735101 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425497 2571 flags.go:64] FLAG: --cluster-domain="" Jul 16 16:50:47.735101 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425500 2571 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 16 16:50:47.735101 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425503 2571 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 16 16:50:47.735101 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425506 2571 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 16 16:50:47.735101 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425509 2571 flags.go:64] FLAG: --container-log-max-files="5" Jul 16 16:50:47.735101 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425513 2571 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 16 16:50:47.735101 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425516 2571 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 16 16:50:47.735101 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425519 2571 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 16 16:50:47.735101 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425522 2571 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 16 16:50:47.735101 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425525 2571 flags.go:64] FLAG: --contention-profiling="false" Jul 16 16:50:47.735101 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425528 2571 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 16 16:50:47.735101 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425530 2571 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 16 16:50:47.735101 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425533 2571 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 16 16:50:47.735766 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425537 2571 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 16 16:50:47.735766 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425541 2571 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 16 16:50:47.735766 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425544 2571 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 16 16:50:47.735766 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425547 2571 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 16 16:50:47.735766 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425550 2571 flags.go:64] FLAG: --enable-load-reader="false" Jul 16 16:50:47.735766 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425553 2571 flags.go:64] FLAG: --enable-server="true" Jul 16 16:50:47.735766 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425556 2571 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 16 16:50:47.735766 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425560 2571 flags.go:64] FLAG: --event-burst="100" Jul 16 16:50:47.735766 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425565 2571 flags.go:64] FLAG: --event-qps="50" Jul 16 16:50:47.735766 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425568 2571 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 16 16:50:47.735766 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425571 2571 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 16 16:50:47.735766 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425574 2571 flags.go:64] FLAG: --eviction-hard="" Jul 16 16:50:47.735766 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425578 2571 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 16 16:50:47.735766 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425580 2571 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 16 16:50:47.735766 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425583 2571 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 16 16:50:47.735766 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425586 2571 flags.go:64] FLAG: --eviction-soft="" Jul 16 16:50:47.735766 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425589 2571 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 16 16:50:47.735766 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425592 2571 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 16 16:50:47.735766 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425595 2571 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 16 16:50:47.735766 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425597 2571 flags.go:64] FLAG: --experimental-mounter-path="" Jul 16 16:50:47.735766 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425600 2571 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 16 16:50:47.735766 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425603 2571 flags.go:64] FLAG: --fail-swap-on="true" Jul 16 16:50:47.735766 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425606 2571 flags.go:64] FLAG: --feature-gates="" Jul 16 16:50:47.735766 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425609 2571 flags.go:64] FLAG: --file-check-frequency="20s" Jul 16 16:50:47.735766 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425612 2571 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 16 16:50:47.736614 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425615 2571 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 16 16:50:47.736614 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425618 2571 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 16 16:50:47.736614 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425621 2571 flags.go:64] FLAG: --healthz-port="10248" Jul 16 16:50:47.736614 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425624 2571 flags.go:64] FLAG: --help="false" Jul 16 16:50:47.736614 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425626 2571 flags.go:64] FLAG: --hostname-override="ip-10-0-128-50.ec2.internal" Jul 16 16:50:47.736614 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425629 2571 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 16 16:50:47.736614 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425632 2571 flags.go:64] FLAG: --http-check-frequency="20s" Jul 16 16:50:47.736614 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425636 2571 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 16 16:50:47.736614 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425639 2571 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 16 16:50:47.736614 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425643 2571 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 16 16:50:47.736614 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425645 2571 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 16 16:50:47.736614 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425648 2571 flags.go:64] FLAG: --image-service-endpoint="" Jul 16 16:50:47.736614 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425651 2571 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 16 16:50:47.736614 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425653 2571 flags.go:64] FLAG: --kube-api-burst="100" Jul 16 16:50:47.736614 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425657 2571 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 16 16:50:47.736614 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425661 2571 flags.go:64] FLAG: --kube-api-qps="50" Jul 16 16:50:47.736614 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425664 2571 flags.go:64] FLAG: --kube-reserved="" Jul 16 16:50:47.736614 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425667 2571 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 16 16:50:47.736614 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425670 2571 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 16 16:50:47.736614 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425673 2571 flags.go:64] FLAG: --kubelet-cgroups="" Jul 16 16:50:47.736614 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425676 2571 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 16 16:50:47.736614 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425678 2571 flags.go:64] FLAG: --lock-file="" Jul 16 16:50:47.736614 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425681 2571 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 16 16:50:47.736614 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425684 2571 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 16 16:50:47.739775 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425687 2571 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 16 16:50:47.739775 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425692 2571 flags.go:64] FLAG: --log-json-split-stream="false" Jul 16 16:50:47.739775 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425695 2571 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 16 16:50:47.739775 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425697 2571 flags.go:64] FLAG: --log-text-split-stream="false" Jul 16 16:50:47.739775 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425700 2571 flags.go:64] FLAG: --logging-format="text" Jul 16 16:50:47.739775 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425703 2571 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 16 16:50:47.739775 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425706 2571 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 16 16:50:47.739775 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425709 2571 flags.go:64] FLAG: --manifest-url="" Jul 16 16:50:47.739775 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425711 2571 flags.go:64] FLAG: --manifest-url-header="" Jul 16 16:50:47.739775 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425715 2571 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 16 16:50:47.739775 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425718 2571 flags.go:64] FLAG: --max-open-files="1000000" Jul 16 16:50:47.739775 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425722 2571 flags.go:64] FLAG: --max-pods="110" Jul 16 16:50:47.739775 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425725 2571 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 16 16:50:47.739775 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425728 2571 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 16 16:50:47.739775 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425731 2571 flags.go:64] FLAG: --memory-manager-policy="None" Jul 16 16:50:47.739775 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425746 2571 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 16 16:50:47.739775 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425749 2571 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 16 16:50:47.739775 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425752 2571 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 16 16:50:47.739775 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425755 2571 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 16 16:50:47.739775 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425762 2571 flags.go:64] FLAG: --node-status-max-images="50" Jul 16 16:50:47.739775 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425765 2571 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 16 16:50:47.739775 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425767 2571 flags.go:64] FLAG: --oom-score-adj="-999" Jul 16 16:50:47.739775 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425770 2571 flags.go:64] FLAG: --pod-cidr="" Jul 16 16:50:47.740569 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425775 2571 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 16 16:50:47.740569 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425780 2571 flags.go:64] FLAG: --pod-manifest-path="" Jul 16 16:50:47.740569 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425783 2571 flags.go:64] FLAG: --pod-max-pids="-1" Jul 16 16:50:47.740569 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425786 2571 flags.go:64] FLAG: --pods-per-core="0" Jul 16 16:50:47.740569 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425789 2571 flags.go:64] FLAG: --port="10250" Jul 16 16:50:47.740569 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425792 2571 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 16 16:50:47.740569 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425795 2571 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0c5ccbf523650ea28" Jul 16 16:50:47.740569 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425798 2571 flags.go:64] FLAG: --qos-reserved="" Jul 16 16:50:47.740569 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425800 2571 flags.go:64] FLAG: --read-only-port="10255" Jul 16 16:50:47.740569 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425803 2571 flags.go:64] FLAG: --register-node="true" Jul 16 16:50:47.740569 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425807 2571 flags.go:64] FLAG: --register-with-taints="" Jul 16 16:50:47.740569 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425813 2571 flags.go:64] FLAG: --registry-burst="10" Jul 16 16:50:47.740569 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425815 2571 flags.go:64] FLAG: --registry-qps="5" Jul 16 16:50:47.740569 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425818 2571 flags.go:64] FLAG: --reserved-cpus="" Jul 16 16:50:47.740569 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425821 2571 flags.go:64] FLAG: --reserved-memory="" Jul 16 16:50:47.740569 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425824 2571 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 16 16:50:47.740569 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425827 2571 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 16 16:50:47.740569 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425830 2571 flags.go:64] FLAG: --rotate-certificates="false" Jul 16 16:50:47.740569 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425833 2571 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 16 16:50:47.740569 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425836 2571 flags.go:64] FLAG: --runonce="false" Jul 16 16:50:47.740569 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425839 2571 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 16 16:50:47.740569 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425841 2571 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 16 16:50:47.740569 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425844 2571 flags.go:64] FLAG: --seccomp-default="false" Jul 16 16:50:47.740569 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425847 2571 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 16 16:50:47.740569 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425851 2571 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 16 16:50:47.742908 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425854 2571 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 16 16:50:47.742908 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425857 2571 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 16 16:50:47.742908 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425860 2571 flags.go:64] FLAG: --storage-driver-password="root" Jul 16 16:50:47.742908 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425863 2571 flags.go:64] FLAG: --storage-driver-secure="false" Jul 16 16:50:47.742908 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425865 2571 flags.go:64] FLAG: --storage-driver-table="stats" Jul 16 16:50:47.742908 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425868 2571 flags.go:64] FLAG: --storage-driver-user="root" Jul 16 16:50:47.742908 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425871 2571 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 16 16:50:47.742908 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425874 2571 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 16 16:50:47.742908 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425879 2571 flags.go:64] FLAG: --system-cgroups="" Jul 16 16:50:47.742908 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425881 2571 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 16 16:50:47.742908 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425886 2571 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 16 16:50:47.742908 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425889 2571 flags.go:64] FLAG: --tls-cert-file="" Jul 16 16:50:47.742908 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425892 2571 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 16 16:50:47.742908 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425896 2571 flags.go:64] FLAG: --tls-min-version="" Jul 16 16:50:47.742908 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425898 2571 flags.go:64] FLAG: --tls-private-key-file="" Jul 16 16:50:47.742908 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425901 2571 flags.go:64] FLAG: --topology-manager-policy="none" Jul 16 16:50:47.742908 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425904 2571 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 16 16:50:47.742908 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425907 2571 flags.go:64] FLAG: --topology-manager-scope="container" Jul 16 16:50:47.742908 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425910 2571 flags.go:64] FLAG: --v="2" Jul 16 16:50:47.742908 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425913 2571 flags.go:64] FLAG: --version="false" Jul 16 16:50:47.742908 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425917 2571 flags.go:64] FLAG: --vmodule="" Jul 16 16:50:47.742908 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425921 2571 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 16 16:50:47.742908 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.425924 2571 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 16 16:50:47.742908 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426028 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 16:50:47.742908 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426032 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 16:50:47.743615 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426035 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 16:50:47.743615 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426038 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 16:50:47.743615 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426041 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 16:50:47.743615 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426043 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 16:50:47.743615 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426046 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 16:50:47.743615 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426048 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 16:50:47.743615 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426052 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 16:50:47.743615 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426054 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 16:50:47.743615 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426057 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 16:50:47.743615 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426060 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 16:50:47.743615 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426062 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 16:50:47.743615 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426064 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 16:50:47.743615 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426067 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 16:50:47.743615 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426070 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 16:50:47.743615 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426073 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 16:50:47.743615 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426076 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 16:50:47.743615 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426079 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 16:50:47.743615 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426081 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 16:50:47.743615 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426084 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 16:50:47.743615 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426087 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 16:50:47.744146 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426089 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 16:50:47.744146 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426092 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 16:50:47.744146 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426094 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 16:50:47.744146 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426097 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 16:50:47.744146 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426099 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 16:50:47.744146 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426102 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 16:50:47.744146 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426104 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 16:50:47.744146 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426107 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 16:50:47.744146 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426109 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 16 16:50:47.744146 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426112 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 16:50:47.744146 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426115 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 16:50:47.744146 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426117 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 16:50:47.744146 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426119 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 16:50:47.744146 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426123 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 16:50:47.744146 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426127 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 16:50:47.744146 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426130 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 16:50:47.744146 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426132 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 16:50:47.744146 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426135 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 16:50:47.744146 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426139 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 16:50:47.744691 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426142 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 16:50:47.744691 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426145 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 16:50:47.744691 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426147 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 16:50:47.744691 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426150 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 16:50:47.744691 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426153 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 16:50:47.744691 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426155 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 16:50:47.744691 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426158 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 16:50:47.744691 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426160 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 16:50:47.744691 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426164 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 16:50:47.744691 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426167 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 16:50:47.744691 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426169 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 16:50:47.744691 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426172 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 16:50:47.744691 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426175 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 16:50:47.744691 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426177 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 16:50:47.744691 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426179 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 16:50:47.744691 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426182 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 16:50:47.744691 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426185 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 16:50:47.744691 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426188 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 16:50:47.744691 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426190 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 16:50:47.744691 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426193 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 16:50:47.745221 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426204 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 16:50:47.745221 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426206 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 16:50:47.745221 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426209 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 16:50:47.745221 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426211 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 16:50:47.745221 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426214 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 16:50:47.745221 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426216 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 16:50:47.745221 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426219 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 16:50:47.745221 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426221 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 16:50:47.745221 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426223 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 16:50:47.745221 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426226 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 16:50:47.745221 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426229 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 16:50:47.745221 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426232 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 16:50:47.745221 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426235 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 16:50:47.745221 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426237 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 16:50:47.745221 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426240 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 16:50:47.745221 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426242 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 16:50:47.745221 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426245 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 16:50:47.745221 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426247 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 16:50:47.745221 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426250 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 16:50:47.830672 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426252 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 16:50:47.830672 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426256 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 16:50:47.830672 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426259 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 16:50:47.830672 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426261 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 16:50:47.830672 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426264 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 16:50:47.830672 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426267 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 16:50:47.830672 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426269 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 16:50:47.830672 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426272 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 16:50:47.830672 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426274 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 16:50:47.830672 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426279 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 16:50:47.830672 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426283 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 16:50:47.830672 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426287 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 16:50:47.830672 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426290 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 16:50:47.830672 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426293 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 16:50:47.830672 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426296 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 16:50:47.830672 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426298 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 16:50:47.830672 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426301 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 16:50:47.830672 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426304 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 16:50:47.830672 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.426306 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 16:50:47.831271 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.427259 2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 16:50:47.831271 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.433483 2571 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 16 16:50:47.831271 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.433498 2571 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 16 16:50:47.831271 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.433516 2571 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 16 16:50:47.831271 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.433522 2571 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 16 16:50:47.831271 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433577 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 16:50:47.831271 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433582 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 16:50:47.831271 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433585 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 16:50:47.831271 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433588 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 16:50:47.831271 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433592 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 16:50:47.831271 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433595 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 16:50:47.831271 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433597 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 16:50:47.831271 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433600 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 16:50:47.831271 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433603 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 16:50:47.831271 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433605 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 16:50:47.832104 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433608 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 16:50:47.832104 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433611 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 16:50:47.832104 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433613 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 16:50:47.832104 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433615 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 16:50:47.832104 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433618 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 16:50:47.832104 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433621 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 16:50:47.832104 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433623 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 16:50:47.832104 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433626 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 16:50:47.832104 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433628 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 16:50:47.832104 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433630 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 16:50:47.832104 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433634 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 16:50:47.832104 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433636 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 16:50:47.832104 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433639 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 16:50:47.832104 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433641 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 16:50:47.832104 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433644 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 16:50:47.832104 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433647 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 16:50:47.832104 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433649 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 16:50:47.832104 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433652 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 16:50:47.832104 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433654 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 16:50:47.832104 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433657 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 16:50:47.832727 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433660 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 16:50:47.832727 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433663 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 16:50:47.832727 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433665 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 16:50:47.832727 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433667 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 16:50:47.832727 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433670 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 16:50:47.832727 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433672 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 16:50:47.832727 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433674 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 16:50:47.832727 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433677 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 16:50:47.832727 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433679 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 16:50:47.832727 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433682 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 16:50:47.832727 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433685 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 16:50:47.832727 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433687 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 16:50:47.832727 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433689 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 16:50:47.832727 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433692 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 16:50:47.832727 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433694 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 16:50:47.832727 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433697 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 16:50:47.832727 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433699 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 16:50:47.832727 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433701 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 16:50:47.832727 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433704 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 16:50:47.832727 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433706 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 16:50:47.833349 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433709 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 16:50:47.833349 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433711 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 16:50:47.833349 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433714 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 16:50:47.833349 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433717 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 16:50:47.833349 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433719 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 16:50:47.833349 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433722 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 16:50:47.833349 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433725 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 16:50:47.833349 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433727 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 16:50:47.833349 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433730 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 16:50:47.833349 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433732 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 16:50:47.833349 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433750 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 16:50:47.833349 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433754 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 16:50:47.833349 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433759 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 16:50:47.833349 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433763 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 16:50:47.833349 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433765 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 16:50:47.833349 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433768 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 16:50:47.833349 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433770 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 16:50:47.833349 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433775 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 16:50:47.833349 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433778 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 16:50:47.833854 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433781 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 16:50:47.833854 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433783 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 16:50:47.833854 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433786 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 16:50:47.833854 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433788 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 16:50:47.833854 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433792 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 16:50:47.833854 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433796 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 16:50:47.833854 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433798 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 16:50:47.833854 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433801 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 16:50:47.833854 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433804 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 16 16:50:47.833854 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433807 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 16:50:47.833854 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433810 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 16:50:47.833854 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433813 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 16:50:47.833854 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433815 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 16:50:47.833854 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433818 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 16:50:47.833854 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433820 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 16:50:47.833854 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433823 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 16:50:47.833854 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433825 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 16:50:47.833854 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433828 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 16:50:47.833854 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433833 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 16:50:47.833854 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433836 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 16:50:47.834421 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433838 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 16:50:47.834421 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433841 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 16:50:47.834421 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433843 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 16:50:47.834421 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433846 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 16:50:47.834421 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433848 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 16:50:47.834421 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433851 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 16:50:47.834421 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433855 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 16:50:47.834421 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433858 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 16:50:47.834421 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433861 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 16:50:47.834421 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433864 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 16:50:47.834421 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.433870 2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 16:50:47.834421 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433971 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 16:50:47.834421 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433976 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 16:50:47.834421 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433979 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 16:50:47.834821 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433982 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 16:50:47.834821 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433984 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 16:50:47.834821 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433987 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 16:50:47.834821 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433989 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 16:50:47.834821 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433992 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 16:50:47.834821 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433994 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 16:50:47.834821 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433997 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 16:50:47.834821 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.433999 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 16:50:47.834821 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434002 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 16:50:47.834821 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434004 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 16:50:47.834821 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434007 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 16:50:47.834821 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434009 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 16:50:47.834821 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434011 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 16:50:47.834821 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434014 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 16:50:47.834821 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434016 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 16:50:47.834821 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434019 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 16:50:47.834821 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434021 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 16:50:47.834821 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434025 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 16:50:47.834821 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434027 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 16:50:47.834821 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434030 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 16:50:47.835352 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434033 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 16:50:47.835352 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434037 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 16:50:47.835352 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434040 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 16:50:47.835352 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434043 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 16:50:47.835352 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434045 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 16:50:47.835352 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434048 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 16:50:47.835352 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434051 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 16:50:47.835352 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434053 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 16:50:47.835352 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434057 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 16:50:47.835352 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434060 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 16:50:47.835352 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434062 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 16:50:47.835352 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434065 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 16:50:47.835352 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434068 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 16:50:47.835352 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434070 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 16:50:47.835352 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434073 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 16:50:47.835352 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434075 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 16:50:47.835352 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434079 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 16:50:47.835352 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434082 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 16:50:47.835352 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434085 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 16:50:47.835864 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434088 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 16:50:47.835864 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434090 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 16:50:47.835864 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434093 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 16:50:47.835864 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434096 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 16:50:47.835864 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434098 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 16:50:47.835864 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434101 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 16:50:47.835864 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434104 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 16:50:47.835864 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434106 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 16:50:47.835864 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434109 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 16:50:47.835864 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434111 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 16:50:47.835864 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434117 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 16:50:47.835864 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434120 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 16:50:47.835864 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434123 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 16:50:47.835864 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434125 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 16:50:47.835864 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434128 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 16:50:47.835864 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434130 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 16:50:47.835864 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434133 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 16:50:47.835864 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434135 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 16:50:47.835864 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434137 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 16:50:47.835864 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434140 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 16:50:47.836410 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434143 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 16:50:47.836410 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434146 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 16:50:47.836410 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434149 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 16:50:47.836410 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434151 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 16:50:47.836410 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434154 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 16:50:47.836410 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434156 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 16:50:47.836410 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434159 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 16:50:47.836410 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434161 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 16:50:47.836410 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434164 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 16:50:47.836410 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434166 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 16 16:50:47.836410 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434168 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 16:50:47.836410 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434171 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 16:50:47.836410 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434173 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 16:50:47.836410 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434175 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 16:50:47.836410 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434178 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 16:50:47.836410 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434180 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 16:50:47.836410 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434183 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 16:50:47.836410 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434185 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 16:50:47.836410 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434188 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 16:50:47.836410 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434190 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 16:50:47.836954 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434193 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 16:50:47.836954 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434203 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 16:50:47.836954 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434208 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 16:50:47.836954 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434211 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 16:50:47.836954 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434214 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 16:50:47.836954 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434216 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 16:50:47.836954 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434219 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 16:50:47.836954 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434221 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 16:50:47.836954 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434224 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 16:50:47.836954 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434227 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 16:50:47.836954 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434229 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 16:50:47.836954 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434232 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 16:50:47.836954 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434235 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 16:50:47.836954 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434239 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 16:50:47.836954 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434242 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 16:50:47.836954 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434244 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 16:50:47.836954 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:47.434247 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 16:50:47.837414 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.434251 2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 16:50:47.837414 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.434340 2571 server.go:962] "Client rotation is on, will bootstrap in background" Jul 16 16:50:47.837414 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.437462 2571 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 16 16:50:47.837414 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.440186 2571 server.go:1019] "Starting client certificate rotation" Jul 16 16:50:47.837414 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.441122 2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 16:50:47.837414 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.442210 2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 16:50:47.837414 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.444233 2571 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 16:50:47.837414 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.449227 2571 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 16:50:47.837414 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.463647 2571 log.go:25] "Validated CRI v1 runtime API" Jul 16 16:50:47.837414 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.466020 2571 log.go:25] "Validated CRI v1 image API" Jul 16 16:50:47.837414 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.468772 2571 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 16 16:50:47.837414 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.471324 2571 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 ac4eeb16-999c-4c3a-b36c-c67aab807790:/dev/nvme0n1p3 ca180291-f49b-4721-948f-3a791fda85f9:/dev/nvme0n1p4] Jul 16 16:50:47.837754 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.471339 2571 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 16 16:50:47.837754 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.472969 2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 16:50:47.838185 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.477029 2571 manager.go:216] Machine: {Timestamp:2026-07-16 16:50:47.475629747 +0000 UTC m=+0.376194020 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3196369 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2104c233ec77ef79a1b67180a91555 SystemUUID:ec2104c2-33ec-77ef-79a1-b67180a91555 BootID:8518e6f7-d1b6-4081-840d-92aaf5e8bf62 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:3c:14:ff:02:e7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:3c:14:ff:02:e7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:82:14:6e:a3:8c:54 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 16 16:50:47.838185 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.477212 2571 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 16 16:50:47.838185 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.477334 2571 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 16 16:50:47.838185 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.480755 2571 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 16 16:50:47.838185 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.480783 2571 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-50.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 16 16:50:47.838185 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.481580 2571 topology_manager.go:138] "Creating topology manager with none policy" Jul 16 16:50:47.838185 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.481589 2571 container_manager_linux.go:309] "Creating device plugin manager" Jul 16 16:50:47.838185 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.481596 2571 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 16:50:47.838185 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.481609 2571 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 16:50:47.838185 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.482811 2571 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 16 16:50:47.838185 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.487340 2571 state_mem.go:36] "Initialized new in-memory state store" Jul 16 16:50:47.838185 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.487448 2571 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 16 16:50:47.838185 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.487478 2571 kubelet.go:486] "Attempting to sync node with API server" Jul 16 16:50:47.838185 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.487496 2571 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 16 16:50:47.838185 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.487512 2571 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 16 16:50:47.838185 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.487521 2571 kubelet.go:398] "Adding apiserver pod source" Jul 16 16:50:47.838185 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.487529 2571 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 16 16:50:47.838185 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.487800 2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.487814 2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.491431 2571 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.493255 2571 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.493282 2571 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.493951 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.493963 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.493969 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.493975 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.493982 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.493987 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.494139 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.494145 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.494151 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.494156 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.494163 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.494171 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.495048 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.495054 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.497761 2571 server.go:1268] "Started kubelet" Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.497806 2571 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.497856 2571 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.497903 2571 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.499392 2571 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.499546 2571 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-50.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:47.500368 2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:47.500368 2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-50.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.502224 2571 server.go:310] "Adding debug handlers to kubelet server" Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:47.503325 2571 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.505299 2571 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.505304 2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:47.505396 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-50.ec2.internal\" not found" Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.506013 2571 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.506023 2571 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.506036 2571 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.506059 2571 factory.go:55] Registering systemd factory Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.506077 2571 factory.go:223] Registration of the systemd container factory successfully Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:47.504970 2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-50.ec2.internal.18c2d339c70369bf default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-50.ec2.internal,UID:ip-10-0-128-50.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-50.ec2.internal,},FirstTimestamp:2026-07-16 16:50:47.497623999 +0000 UTC m=+0.398188327,LastTimestamp:2026-07-16 16:50:47.497623999 +0000 UTC m=+0.398188327,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-50.ec2.internal,}" Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.506183 2571 reconstruct.go:97] "Volume reconstruction finished" Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.506191 2571 reconciler.go:29] "Reconciler: start to sync state" Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.506249 2571 factory.go:153] Registering CRI-O factory Jul 16 16:50:47.838770 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.506264 2571 factory.go:223] Registration of the crio container factory successfully Jul 16 16:50:47.908081 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.506323 2571 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 16 16:50:47.908081 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.506346 2571 factory.go:103] Registering Raw factory Jul 16 16:50:47.908081 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.506360 2571 manager.go:1196] Started watching for new ooms in manager Jul 16 16:50:47.908081 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.506731 2571 manager.go:349] Starting recovery of all containers Jul 16 16:50:47.908081 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:47.513086 2571 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 16 16:50:47.908081 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:47.513201 2571 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-50.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 16 16:50:47.908081 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.519228 2571 manager.go:354] Recovery completed Jul 16 16:50:47.908081 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.524119 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 16:50:47.908081 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.525812 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientMemory" Jul 16 16:50:47.908081 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.525835 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 16:50:47.908081 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.525854 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientPID" Jul 16 16:50:47.908081 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.526246 2571 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 16 16:50:47.908081 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.526254 2571 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 16 16:50:47.908081 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.526271 2571 state_mem.go:36] "Initialized new in-memory state store" Jul 16 16:50:47.908081 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.526273 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 16:50:47.908081 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.526882 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientMemory" Jul 16 16:50:47.908081 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.526909 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 16:50:47.908081 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.526921 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientPID" Jul 16 16:50:47.908081 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:47.527829 2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-50.ec2.internal.18c2d339c8b1b455 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-50.ec2.internal,UID:ip-10-0-128-50.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-50.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-50.ec2.internal,},FirstTimestamp:2026-07-16 16:50:47.525823573 +0000 UTC m=+0.426387846,LastTimestamp:2026-07-16 16:50:47.525823573 +0000 UTC m=+0.426387846,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-50.ec2.internal,}" Jul 16 16:50:47.908081 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.528336 2571 policy_none.go:49] "None policy: Start" Jul 16 16:50:47.908081 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.528349 2571 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 16 16:50:47.908081 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.528358 2571 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 16 16:50:47.908081 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.530338 2571 policy_none.go:47] "Start" Jul 16 16:50:47.908081 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:47.537189 2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-50.ec2.internal.18c2d339c8b2196e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-50.ec2.internal,UID:ip-10-0-128-50.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-50.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-50.ec2.internal,},FirstTimestamp:2026-07-16 16:50:47.525849454 +0000 UTC m=+0.426413728,LastTimestamp:2026-07-16 16:50:47.525849454 +0000 UTC m=+0.426413728,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-50.ec2.internal,}" Jul 16 16:50:47.908081 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:47.547826 2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-50.ec2.internal.18c2d339c8b23b70 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-50.ec2.internal,UID:ip-10-0-128-50.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-50.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-50.ec2.internal,},FirstTimestamp:2026-07-16 16:50:47.52585816 +0000 UTC m=+0.426422435,LastTimestamp:2026-07-16 16:50:47.52585816 +0000 UTC m=+0.426422435,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-50.ec2.internal,}" Jul 16 16:50:47.908922 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:47.557164 2571 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-50.ec2.internal.18c2d339c8b1b455\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-50.ec2.internal.18c2d339c8b1b455 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-50.ec2.internal,UID:ip-10-0-128-50.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-50.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-50.ec2.internal,},FirstTimestamp:2026-07-16 16:50:47.525823573 +0000 UTC m=+0.426387846,LastTimestamp:2026-07-16 16:50:47.526893936 +0000 UTC m=+0.427458210,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-50.ec2.internal,}" Jul 16 16:50:47.908922 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:47.566556 2571 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-50.ec2.internal.18c2d339c8b2196e\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-50.ec2.internal.18c2d339c8b2196e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-50.ec2.internal,UID:ip-10-0-128-50.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-50.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-50.ec2.internal,},FirstTimestamp:2026-07-16 16:50:47.525849454 +0000 UTC m=+0.426413728,LastTimestamp:2026-07-16 16:50:47.526915677 +0000 UTC m=+0.427479950,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-50.ec2.internal,}" Jul 16 16:50:47.908922 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:47.579112 2571 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-50.ec2.internal.18c2d339c8b23b70\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-50.ec2.internal.18c2d339c8b23b70 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-50.ec2.internal,UID:ip-10-0-128-50.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-50.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-50.ec2.internal,},FirstTimestamp:2026-07-16 16:50:47.52585816 +0000 UTC m=+0.426422435,LastTimestamp:2026-07-16 16:50:47.526925226 +0000 UTC m=+0.427489500,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-50.ec2.internal,}" Jul 16 16:50:47.908922 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.599282 2571 manager.go:337] "Starting Device Plugin manager" Jul 16 16:50:47.908922 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:47.599338 2571 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 16 16:50:47.908922 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.599350 2571 server.go:87] "Starting device plugin registration server" Jul 16 16:50:47.908922 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.599547 2571 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 16 16:50:47.908922 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.599557 2571 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 16 16:50:47.908922 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.599613 2571 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 16 16:50:47.908922 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.599698 2571 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 16 16:50:47.908922 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.599707 2571 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 16 16:50:47.908922 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:47.600121 2571 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 16 16:50:47.908922 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:47.600164 2571 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-50.ec2.internal\" not found" Jul 16 16:50:47.908922 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.610519 2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 16 16:50:47.908922 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:47.611537 2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-50.ec2.internal.18c2d339cd34972f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-50.ec2.internal,UID:ip-10-0-128-50.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-128-50.ec2.internal,},FirstTimestamp:2026-07-16 16:50:47.601510191 +0000 UTC m=+0.502074463,LastTimestamp:2026-07-16 16:50:47.601510191 +0000 UTC m=+0.502074463,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-50.ec2.internal,}" Jul 16 16:50:47.908922 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.611718 2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 16 16:50:47.908922 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.611766 2571 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 16 16:50:47.908922 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.611783 2571 kubelet.go:2448] "Starting kubelet main sync loop" Jul 16 16:50:47.908922 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:47.611813 2571 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 16 16:50:47.909539 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:47.620885 2571 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" Jul 16 16:50:47.909539 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.664847 2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6zltx" Jul 16 16:50:47.909539 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.673262 2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6zltx" Jul 16 16:50:47.909539 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.700573 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 16:50:47.909539 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.701213 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientMemory" Jul 16 16:50:47.909539 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.701238 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 16:50:47.909539 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.701249 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientPID" Jul 16 16:50:47.909539 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.701273 2571 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-50.ec2.internal" Jul 16 16:50:47.909539 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.712251 2571 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-128-50.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal"] Jul 16 16:50:47.909539 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.712296 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 16:50:47.909539 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.713588 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientMemory" Jul 16 16:50:47.909539 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.713609 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 16:50:47.909539 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.713617 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientPID" Jul 16 16:50:47.909539 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.715831 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 16:50:47.909539 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.715932 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.ec2.internal" Jul 16 16:50:47.909539 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.715954 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 16:50:47.909539 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.717365 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientMemory" Jul 16 16:50:47.909539 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.717386 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 16:50:47.909539 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.717398 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientPID" Jul 16 16:50:47.909539 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.717437 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientMemory" Jul 16 16:50:47.909539 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.717453 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 16:50:47.909539 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.717461 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientPID" Jul 16 16:50:47.909539 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.719587 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal" Jul 16 16:50:47.909539 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.719608 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 16:50:47.909539 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.720279 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientMemory" Jul 16 16:50:47.909539 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.720304 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 16:50:47.909539 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.720317 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientPID" Jul 16 16:50:47.909539 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.720533 2571 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-50.ec2.internal" Jul 16 16:50:47.909539 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:47.720555 2571 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-50.ec2.internal\": node \"ip-10-0-128-50.ec2.internal\" not found" Jul 16 16:50:47.909539 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:47.746325 2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-50.ec2.internal\" not found" node="ip-10-0-128-50.ec2.internal" Jul 16 16:50:47.909539 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:47.748489 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-50.ec2.internal\" not found" Jul 16 16:50:47.909539 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:47.750456 2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-50.ec2.internal\" not found" node="ip-10-0-128-50.ec2.internal" Jul 16 16:50:47.909539 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.808610 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b70782e48d7de5f190e2cd3c95d4336b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal\" (UID: \"b70782e48d7de5f190e2cd3c95d4336b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal" Jul 16 16:50:47.909539 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.808633 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b70782e48d7de5f190e2cd3c95d4336b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal\" (UID: \"b70782e48d7de5f190e2cd3c95d4336b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal" Jul 16 16:50:47.910442 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.808654 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/683869ec7e8dd47873bd79d88a644cf6-config\") pod \"kube-apiserver-proxy-ip-10-0-128-50.ec2.internal\" (UID: \"683869ec7e8dd47873bd79d88a644cf6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.ec2.internal" Jul 16 16:50:47.910442 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:47.849444 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-50.ec2.internal\" not found" Jul 16 16:50:47.910442 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.908908 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b70782e48d7de5f190e2cd3c95d4336b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal\" (UID: \"b70782e48d7de5f190e2cd3c95d4336b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal" Jul 16 16:50:47.910442 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.908933 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b70782e48d7de5f190e2cd3c95d4336b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal\" (UID: \"b70782e48d7de5f190e2cd3c95d4336b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal" Jul 16 16:50:47.910442 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.908948 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/683869ec7e8dd47873bd79d88a644cf6-config\") pod \"kube-apiserver-proxy-ip-10-0-128-50.ec2.internal\" (UID: \"683869ec7e8dd47873bd79d88a644cf6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.ec2.internal" Jul 16 16:50:47.910442 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.908992 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b70782e48d7de5f190e2cd3c95d4336b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal\" (UID: \"b70782e48d7de5f190e2cd3c95d4336b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal" Jul 16 16:50:47.910442 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.908997 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/683869ec7e8dd47873bd79d88a644cf6-config\") pod \"kube-apiserver-proxy-ip-10-0-128-50.ec2.internal\" (UID: \"683869ec7e8dd47873bd79d88a644cf6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.ec2.internal" Jul 16 16:50:47.910442 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:47.909017 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b70782e48d7de5f190e2cd3c95d4336b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal\" (UID: \"b70782e48d7de5f190e2cd3c95d4336b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal" Jul 16 16:50:48.016084 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:47.950153 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-50.ec2.internal\" not found" Jul 16 16:50:48.132902 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:48.049728 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.ec2.internal" Jul 16 16:50:48.132902 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:48.050786 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-50.ec2.internal\" not found" Jul 16 16:50:48.132902 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:48.054561 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal" Jul 16 16:50:48.236151 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:48.150963 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-50.ec2.internal\" not found" Jul 16 16:50:48.341483 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:48.251372 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-50.ec2.internal\" not found" Jul 16 16:50:48.471696 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:48.351864 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-50.ec2.internal\" not found" Jul 16 16:50:48.471696 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:48.439333 2571 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 16 16:50:48.471696 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:48.452597 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-50.ec2.internal\" not found" Jul 16 16:50:49.744661 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:49.700265 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 16:50:49.744661 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:49.701797 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientMemory" Jul 16 16:50:49.744661 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:49.701822 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 16:50:49.744661 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:49.701831 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientPID" Jul 16 16:50:49.744661 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:49.710125 2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 16:50:49.744661 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:49.711777 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-50.ec2.internal\" not found" Jul 16 16:50:49.851591 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:49.766834 2571 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 16:50:49.851591 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:49.786791 2571 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 16:50:49.851591 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:49.839440 2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-15 16:45:48 +0000 UTC" deadline="2028-01-30 22:36:47.27122795 +0000 UTC" Jul 16 16:50:49.851591 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:49.839463 2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13517h45m57.431767531s" Jul 16 16:50:49.957257 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:49.864578 2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.ec2.internal" Jul 16 16:50:49.957257 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:49.866377 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod683869ec7e8dd47873bd79d88a644cf6.slice/crio-a11455e2f10993f992dc2f6c1866c6aaf05c8881afc4f55494f971ad8db4f3ff WatchSource:0}: Error finding container a11455e2f10993f992dc2f6c1866c6aaf05c8881afc4f55494f971ad8db4f3ff: Status 404 returned error can't find the container with id a11455e2f10993f992dc2f6c1866c6aaf05c8881afc4f55494f971ad8db4f3ff Jul 16 16:50:49.957257 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:49.871075 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 16:50:49.957257 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:49.875545 2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 16:50:49.957257 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:49.876387 2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal" Jul 16 16:50:49.957257 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:49.890375 2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 16:50:50.193201 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.116162 2571 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 16:50:50.691801 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.647567 2571 apiserver.go:52] "Watching apiserver" Jul 16 16:50:50.691801 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.655381 2571 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 16 16:50:50.691801 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.655711 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-slv5t","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal","openshift-multus/multus-zn2rb","openshift-multus/network-metrics-daemon-4p6nc","openshift-network-diagnostics/network-check-target-zfpzk","openshift-network-operator/iptables-alerter-6wmn9","openshift-ovn-kubernetes/ovnkube-node-qd7bg","kube-system/global-pull-secret-syncer-wd65t","kube-system/konnectivity-agent-df9xm","kube-system/kube-apiserver-proxy-ip-10-0-128-50.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djwkt","openshift-cluster-node-tuning-operator/tuned-dgn58","openshift-dns/node-resolver-fttnf","openshift-image-registry/node-ca-ln8q5"] Jul 16 16:50:50.691801 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.659298 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.691801 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.661975 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-slv5t" Jul 16 16:50:50.691801 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.662361 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-c8b42\"" Jul 16 16:50:50.691801 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.663390 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 16 16:50:50.691801 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.663411 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 16 16:50:50.691801 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.663482 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 16 16:50:50.691801 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.663519 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 16 16:50:50.691801 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.663615 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 16 16:50:50.691801 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.663694 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 16 16:50:50.691801 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.664166 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.691801 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.664696 2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 16 16:50:50.691801 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.664767 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 16 16:50:50.691801 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.665332 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 16 16:50:50.691801 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.666631 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4p6nc" Jul 16 16:50:50.691801 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:50.666689 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4p6nc" podUID="2341bdcc-01b6-426c-8189-71be7fdc2e8b" Jul 16 16:50:50.691801 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.668520 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zfpzk" Jul 16 16:50:50.691801 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:50.668584 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zfpzk" podUID="1c20e16c-0039-4705-a02f-d2a8a5c66ae1" Jul 16 16:50:50.691801 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.675810 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-host-slash\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.691801 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.675840 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-run-openvswitch\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.691801 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.675864 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-run-ovn\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.691801 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.675886 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cfbe9f0b-b759-4064-9332-3b48c547c003-ovn-node-metrics-cert\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.691801 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.675927 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9c75f9a1-e0ed-45af-b4c4-e639989e6d3b-cni-binary-copy\") pod \"multus-additional-cni-plugins-slv5t\" (UID: \"9c75f9a1-e0ed-45af-b4c4-e639989e6d3b\") " pod="openshift-multus/multus-additional-cni-plugins-slv5t" Jul 16 16:50:50.693241 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.675950 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9c75f9a1-e0ed-45af-b4c4-e639989e6d3b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-slv5t\" (UID: \"9c75f9a1-e0ed-45af-b4c4-e639989e6d3b\") " pod="openshift-multus/multus-additional-cni-plugins-slv5t" Jul 16 16:50:50.693241 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.675972 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-node-log\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.693241 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.675993 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cfbe9f0b-b759-4064-9332-3b48c547c003-ovnkube-script-lib\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.693241 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.676018 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9c75f9a1-e0ed-45af-b4c4-e639989e6d3b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-slv5t\" (UID: \"9c75f9a1-e0ed-45af-b4c4-e639989e6d3b\") " pod="openshift-multus/multus-additional-cni-plugins-slv5t" Jul 16 16:50:50.693241 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.676041 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-system-cni-dir\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.693241 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.676068 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-cnibin\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.693241 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.676091 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-multus-socket-dir-parent\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.693241 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.676113 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-host-var-lib-kubelet\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.701093 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.701073 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2341bdcc-01b6-426c-8189-71be7fdc2e8b-metrics-certs\") pod \"network-metrics-daemon-4p6nc\" (UID: \"2341bdcc-01b6-426c-8189-71be7fdc2e8b\") " pod="openshift-multus/network-metrics-daemon-4p6nc" Jul 16 16:50:50.701167 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.701102 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-systemd-units\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.701167 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.701121 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-host-run-netns\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.701167 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.701155 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-host-cni-bin\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.701335 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.701190 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pwkp\" (UniqueName: \"kubernetes.io/projected/cfbe9f0b-b759-4064-9332-3b48c547c003-kube-api-access-6pwkp\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.701335 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.701211 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 16 16:50:50.701335 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.701216 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9c75f9a1-e0ed-45af-b4c4-e639989e6d3b-os-release\") pod \"multus-additional-cni-plugins-slv5t\" (UID: \"9c75f9a1-e0ed-45af-b4c4-e639989e6d3b\") " pod="openshift-multus/multus-additional-cni-plugins-slv5t" Jul 16 16:50:50.701335 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.701259 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 16 16:50:50.701335 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.701271 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-host-run-netns\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.701335 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.701289 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-etc-kubernetes\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.701335 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.701308 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-host-kubelet\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.701335 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.701322 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 16 16:50:50.701335 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.701333 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-run-systemd\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.701781 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.701348 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.701781 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.701365 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-hostroot\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.701781 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.701403 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cfbe9f0b-b759-4064-9332-3b48c547c003-ovnkube-config\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.701781 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.701423 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cfbe9f0b-b759-4064-9332-3b48c547c003-env-overrides\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.701781 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.701437 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9c75f9a1-e0ed-45af-b4c4-e639989e6d3b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-slv5t\" (UID: \"9c75f9a1-e0ed-45af-b4c4-e639989e6d3b\") " pod="openshift-multus/multus-additional-cni-plugins-slv5t" Jul 16 16:50:50.701781 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.701462 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b7428d07-679c-4e1c-bbaf-36230bfb797b-cni-binary-copy\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.701781 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.701476 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 16 16:50:50.701781 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.701488 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-host-var-lib-cni-multus\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.701781 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.701493 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qbkqs\"" Jul 16 16:50:50.701781 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.701519 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-p6zgn\"" Jul 16 16:50:50.701781 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.701549 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-host-cni-netd\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.701781 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.701570 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-os-release\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.701781 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.701583 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-host-run-k8s-cni-cncf-io\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.701781 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.701614 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-host-var-lib-cni-bin\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.701781 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.701643 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ht7hx\" (UniqueName: \"kubernetes.io/projected/b7428d07-679c-4e1c-bbaf-36230bfb797b-kube-api-access-ht7hx\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.701781 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.701674 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-var-lib-openvswitch\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.701781 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.701693 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-etc-openvswitch\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.701781 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.701706 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-host-run-ovn-kubernetes\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.702468 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.701748 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-multus-cni-dir\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.702468 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.701781 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-host-run-multus-certs\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.702468 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.701809 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvjmf\" (UniqueName: \"kubernetes.io/projected/2341bdcc-01b6-426c-8189-71be7fdc2e8b-kube-api-access-dvjmf\") pod \"network-metrics-daemon-4p6nc\" (UID: \"2341bdcc-01b6-426c-8189-71be7fdc2e8b\") " pod="openshift-multus/network-metrics-daemon-4p6nc" Jul 16 16:50:50.702468 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.701850 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-log-socket\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.702468 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.701877 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9c75f9a1-e0ed-45af-b4c4-e639989e6d3b-system-cni-dir\") pod \"multus-additional-cni-plugins-slv5t\" (UID: \"9c75f9a1-e0ed-45af-b4c4-e639989e6d3b\") " pod="openshift-multus/multus-additional-cni-plugins-slv5t" Jul 16 16:50:50.702468 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.701901 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9c75f9a1-e0ed-45af-b4c4-e639989e6d3b-cnibin\") pod \"multus-additional-cni-plugins-slv5t\" (UID: \"9c75f9a1-e0ed-45af-b4c4-e639989e6d3b\") " pod="openshift-multus/multus-additional-cni-plugins-slv5t" Jul 16 16:50:50.702468 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.702352 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sk8f\" (UniqueName: \"kubernetes.io/projected/9c75f9a1-e0ed-45af-b4c4-e639989e6d3b-kube-api-access-9sk8f\") pod \"multus-additional-cni-plugins-slv5t\" (UID: \"9c75f9a1-e0ed-45af-b4c4-e639989e6d3b\") " pod="openshift-multus/multus-additional-cni-plugins-slv5t" Jul 16 16:50:50.702468 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.702416 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-multus-conf-dir\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.702468 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.702444 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b7428d07-679c-4e1c-bbaf-36230bfb797b-multus-daemon-config\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.702892 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.702480 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.704878 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.704855 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 16 16:50:50.705166 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.705128 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fttnf" Jul 16 16:50:50.705245 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.705150 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 16 16:50:50.705641 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.705626 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-lhmpg\"" Jul 16 16:50:50.707715 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.707697 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-df9xm" Jul 16 16:50:50.708061 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.708044 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7c876\"" Jul 16 16:50:50.708144 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.708085 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 16 16:50:50.708144 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.708087 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 16 16:50:50.708271 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.708257 2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 16:50:50.709991 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.709975 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djwkt" Jul 16 16:50:50.710249 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.710221 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 16 16:50:50.710491 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.710473 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-5btxt\"" Jul 16 16:50:50.710567 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.710535 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 16 16:50:50.712198 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.712184 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6wmn9" Jul 16 16:50:50.712337 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.712322 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 16 16:50:50.712492 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.712472 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 16 16:50:50.712658 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.712640 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jjd89\"" Jul 16 16:50:50.712976 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.712963 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 16 16:50:50.714488 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.714469 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wd65t" Jul 16 16:50:50.714565 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:50.714513 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wd65t" podUID="598a7966-d69c-4867-9936-667f0c47ed23" Jul 16 16:50:50.716771 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.716752 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ln8q5" Jul 16 16:50:50.725286 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.724989 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 16 16:50:50.725286 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.725011 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-kzknz\"" Jul 16 16:50:50.725286 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.725061 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 16 16:50:50.725286 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.725088 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-xmqvz\"" Jul 16 16:50:50.725286 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.725219 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 16 16:50:50.731984 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.731953 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 16 16:50:50.732075 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.731996 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 16 16:50:50.732075 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.732034 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 16 16:50:50.733066 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.733048 2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2gtff" Jul 16 16:50:50.746838 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.746823 2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2gtff" Jul 16 16:50:50.766551 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.766536 2571 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 16 16:50:50.774224 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.774195 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.ec2.internal" event={"ID":"683869ec7e8dd47873bd79d88a644cf6","Type":"ContainerStarted","Data":"a11455e2f10993f992dc2f6c1866c6aaf05c8881afc4f55494f971ad8db4f3ff"} Jul 16 16:50:50.803538 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.803518 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-os-release\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.803591 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.803553 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-host-var-lib-cni-bin\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.803591 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.803568 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ht7hx\" (UniqueName: \"kubernetes.io/projected/b7428d07-679c-4e1c-bbaf-36230bfb797b-kube-api-access-ht7hx\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.803675 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.803589 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a686a394-6813-4075-8118-9c99e4c189f2-tmp\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.803675 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.803611 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-etc-openvswitch\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.803675 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.803633 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-multus-cni-dir\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.803675 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.803655 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-os-release\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.803675 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.803665 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-etc-openvswitch\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.803918 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.803655 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a686a394-6813-4075-8118-9c99e4c189f2-etc-modprobe-d\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.803918 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.803705 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7e16013-94ca-4a61-b551-f3f3b66d3cb1-tmp-dir\") pod \"node-resolver-fttnf\" (UID: \"f7e16013-94ca-4a61-b551-f3f3b66d3cb1\") " pod="openshift-dns/node-resolver-fttnf" Jul 16 16:50:50.803918 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.803735 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9c75f9a1-e0ed-45af-b4c4-e639989e6d3b-system-cni-dir\") pod \"multus-additional-cni-plugins-slv5t\" (UID: \"9c75f9a1-e0ed-45af-b4c4-e639989e6d3b\") " pod="openshift-multus/multus-additional-cni-plugins-slv5t" Jul 16 16:50:50.803918 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.803749 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9c75f9a1-e0ed-45af-b4c4-e639989e6d3b-cnibin\") pod \"multus-additional-cni-plugins-slv5t\" (UID: \"9c75f9a1-e0ed-45af-b4c4-e639989e6d3b\") " pod="openshift-multus/multus-additional-cni-plugins-slv5t" Jul 16 16:50:50.803918 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.803767 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b7428d07-679c-4e1c-bbaf-36230bfb797b-multus-daemon-config\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.803918 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.803679 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-host-var-lib-cni-bin\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.803918 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.803807 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9c75f9a1-e0ed-45af-b4c4-e639989e6d3b-cnibin\") pod \"multus-additional-cni-plugins-slv5t\" (UID: \"9c75f9a1-e0ed-45af-b4c4-e639989e6d3b\") " pod="openshift-multus/multus-additional-cni-plugins-slv5t" Jul 16 16:50:50.803918 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.803810 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1ade0d05-b097-449c-8207-734ba764bf02-registration-dir\") pod \"aws-ebs-csi-driver-node-djwkt\" (UID: \"1ade0d05-b097-449c-8207-734ba764bf02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djwkt" Jul 16 16:50:50.803918 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.803822 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9c75f9a1-e0ed-45af-b4c4-e639989e6d3b-system-cni-dir\") pod \"multus-additional-cni-plugins-slv5t\" (UID: \"9c75f9a1-e0ed-45af-b4c4-e639989e6d3b\") " pod="openshift-multus/multus-additional-cni-plugins-slv5t" Jul 16 16:50:50.803918 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.803831 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-multus-cni-dir\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.803918 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.803840 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a686a394-6813-4075-8118-9c99e4c189f2-etc-kubernetes\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.803918 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.803871 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a686a394-6813-4075-8118-9c99e4c189f2-etc-sysctl-d\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.803918 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.803886 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a686a394-6813-4075-8118-9c99e4c189f2-lib-modules\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.803918 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.803900 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/598a7966-d69c-4867-9936-667f0c47ed23-kubelet-config\") pod \"global-pull-secret-syncer-wd65t\" (UID: \"598a7966-d69c-4867-9936-667f0c47ed23\") " pod="kube-system/global-pull-secret-syncer-wd65t" Jul 16 16:50:50.803918 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.803916 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-run-openvswitch\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.804443 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.803967 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-run-openvswitch\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.804443 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.804245 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-run-ovn\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.804443 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.803970 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-run-ovn\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.804443 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.804299 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9c75f9a1-e0ed-45af-b4c4-e639989e6d3b-cni-binary-copy\") pod \"multus-additional-cni-plugins-slv5t\" (UID: \"9c75f9a1-e0ed-45af-b4c4-e639989e6d3b\") " pod="openshift-multus/multus-additional-cni-plugins-slv5t" Jul 16 16:50:50.804443 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.804334 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-cnibin\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.804443 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.804363 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-host-var-lib-kubelet\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.804443 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.804393 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2341bdcc-01b6-426c-8189-71be7fdc2e8b-metrics-certs\") pod \"network-metrics-daemon-4p6nc\" (UID: \"2341bdcc-01b6-426c-8189-71be7fdc2e8b\") " pod="openshift-multus/network-metrics-daemon-4p6nc" Jul 16 16:50:50.804775 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.804462 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-cnibin\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.804775 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.804505 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1ade0d05-b097-449c-8207-734ba764bf02-kubelet-dir\") pod \"aws-ebs-csi-driver-node-djwkt\" (UID: \"1ade0d05-b097-449c-8207-734ba764bf02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djwkt" Jul 16 16:50:50.804775 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.804627 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-node-log\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.804775 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.804692 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cfbe9f0b-b759-4064-9332-3b48c547c003-ovnkube-script-lib\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.804775 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:50.804703 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 16:50:50.804775 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.804744 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-system-cni-dir\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.805041 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.804784 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r9xl\" (UniqueName: \"kubernetes.io/projected/1ade0d05-b097-449c-8207-734ba764bf02-kube-api-access-9r9xl\") pod \"aws-ebs-csi-driver-node-djwkt\" (UID: \"1ade0d05-b097-449c-8207-734ba764bf02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djwkt" Jul 16 16:50:50.805041 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:50.804822 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2341bdcc-01b6-426c-8189-71be7fdc2e8b-metrics-certs podName:2341bdcc-01b6-426c-8189-71be7fdc2e8b nodeName:}" failed. No retries permitted until 2026-07-16 16:50:51.304797016 +0000 UTC m=+3.046372301 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2341bdcc-01b6-426c-8189-71be7fdc2e8b-metrics-certs") pod "network-metrics-daemon-4p6nc" (UID: "2341bdcc-01b6-426c-8189-71be7fdc2e8b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 16:50:50.805041 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.804864 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b7428d07-679c-4e1c-bbaf-36230bfb797b-multus-daemon-config\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.805041 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.804871 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-host-var-lib-kubelet\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.805041 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.804888 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-system-cni-dir\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.805041 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.804951 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-node-log\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.805041 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.804950 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1dd02f2a-342f-46a3-bbba-b630378fa3db-host\") pod \"node-ca-ln8q5\" (UID: \"1dd02f2a-342f-46a3-bbba-b630378fa3db\") " pod="openshift-image-registry/node-ca-ln8q5" Jul 16 16:50:50.805041 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.804995 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a686a394-6813-4075-8118-9c99e4c189f2-etc-systemd\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.805406 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.805080 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f7e16013-94ca-4a61-b551-f3f3b66d3cb1-hosts-file\") pod \"node-resolver-fttnf\" (UID: \"f7e16013-94ca-4a61-b551-f3f3b66d3cb1\") " pod="openshift-dns/node-resolver-fttnf" Jul 16 16:50:50.805406 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.805106 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-systemd-units\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.805406 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.805124 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9c75f9a1-e0ed-45af-b4c4-e639989e6d3b-cni-binary-copy\") pod \"multus-additional-cni-plugins-slv5t\" (UID: \"9c75f9a1-e0ed-45af-b4c4-e639989e6d3b\") " pod="openshift-multus/multus-additional-cni-plugins-slv5t" Jul 16 16:50:50.805406 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.805147 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-host-run-netns\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.805406 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.805194 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-host-cni-bin\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.805406 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.805223 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-systemd-units\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.805406 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.805234 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-host-cni-bin\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.805406 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.805232 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-host-run-netns\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.805406 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.805290 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9c75f9a1-e0ed-45af-b4c4-e639989e6d3b-os-release\") pod \"multus-additional-cni-plugins-slv5t\" (UID: \"9c75f9a1-e0ed-45af-b4c4-e639989e6d3b\") " pod="openshift-multus/multus-additional-cni-plugins-slv5t" Jul 16 16:50:50.805406 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.805324 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-etc-kubernetes\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.805406 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.805358 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1ade0d05-b097-449c-8207-734ba764bf02-socket-dir\") pod \"aws-ebs-csi-driver-node-djwkt\" (UID: \"1ade0d05-b097-449c-8207-734ba764bf02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djwkt" Jul 16 16:50:50.805406 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.805368 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9c75f9a1-e0ed-45af-b4c4-e639989e6d3b-os-release\") pod \"multus-additional-cni-plugins-slv5t\" (UID: \"9c75f9a1-e0ed-45af-b4c4-e639989e6d3b\") " pod="openshift-multus/multus-additional-cni-plugins-slv5t" Jul 16 16:50:50.805406 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.805382 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/598a7966-d69c-4867-9936-667f0c47ed23-original-pull-secret\") pod \"global-pull-secret-syncer-wd65t\" (UID: \"598a7966-d69c-4867-9936-667f0c47ed23\") " pod="kube-system/global-pull-secret-syncer-wd65t" Jul 16 16:50:50.805406 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.805402 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-etc-kubernetes\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.805994 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.805457 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-host-kubelet\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.805994 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.805487 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cfbe9f0b-b759-4064-9332-3b48c547c003-env-overrides\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.805994 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.805515 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9c75f9a1-e0ed-45af-b4c4-e639989e6d3b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-slv5t\" (UID: \"9c75f9a1-e0ed-45af-b4c4-e639989e6d3b\") " pod="openshift-multus/multus-additional-cni-plugins-slv5t" Jul 16 16:50:50.805994 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.805525 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-host-kubelet\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.805994 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.805546 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b7428d07-679c-4e1c-bbaf-36230bfb797b-cni-binary-copy\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.805994 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.805622 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1ade0d05-b097-449c-8207-734ba764bf02-etc-selinux\") pod \"aws-ebs-csi-driver-node-djwkt\" (UID: \"1ade0d05-b097-449c-8207-734ba764bf02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djwkt" Jul 16 16:50:50.805994 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.805663 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a686a394-6813-4075-8118-9c99e4c189f2-run\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.805994 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.805663 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9c75f9a1-e0ed-45af-b4c4-e639989e6d3b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-slv5t\" (UID: \"9c75f9a1-e0ed-45af-b4c4-e639989e6d3b\") " pod="openshift-multus/multus-additional-cni-plugins-slv5t" Jul 16 16:50:50.805994 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.805698 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-host-cni-netd\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.805994 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.805767 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-host-cni-netd\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.805994 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.805790 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-host-run-k8s-cni-cncf-io\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.805994 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.805827 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/27c7185c-f0e1-4486-882a-bf9275296975-agent-certs\") pod \"konnectivity-agent-df9xm\" (UID: \"27c7185c-f0e1-4486-882a-bf9275296975\") " pod="kube-system/konnectivity-agent-df9xm" Jul 16 16:50:50.805994 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.805858 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/32c71c9a-59a2-4743-8305-520f13688212-iptables-alerter-script\") pod \"iptables-alerter-6wmn9\" (UID: \"32c71c9a-59a2-4743-8305-520f13688212\") " pod="openshift-network-operator/iptables-alerter-6wmn9" Jul 16 16:50:50.805994 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.805890 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-host-run-k8s-cni-cncf-io\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.805994 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.805906 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-var-lib-openvswitch\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.805994 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.805942 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-host-run-ovn-kubernetes\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.805994 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.805970 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cfbe9f0b-b759-4064-9332-3b48c547c003-env-overrides\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.806707 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.806025 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-host-run-ovn-kubernetes\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.806707 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.806017 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-host-run-multus-certs\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.806707 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.806069 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-var-lib-openvswitch\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.806707 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.806084 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b7428d07-679c-4e1c-bbaf-36230bfb797b-cni-binary-copy\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.806707 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.806096 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvjmf\" (UniqueName: \"kubernetes.io/projected/2341bdcc-01b6-426c-8189-71be7fdc2e8b-kube-api-access-dvjmf\") pod \"network-metrics-daemon-4p6nc\" (UID: \"2341bdcc-01b6-426c-8189-71be7fdc2e8b\") " pod="openshift-multus/network-metrics-daemon-4p6nc" Jul 16 16:50:50.806707 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.806138 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-log-socket\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.806707 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.806170 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sk8f\" (UniqueName: \"kubernetes.io/projected/9c75f9a1-e0ed-45af-b4c4-e639989e6d3b-kube-api-access-9sk8f\") pod \"multus-additional-cni-plugins-slv5t\" (UID: \"9c75f9a1-e0ed-45af-b4c4-e639989e6d3b\") " pod="openshift-multus/multus-additional-cni-plugins-slv5t" Jul 16 16:50:50.806707 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.806177 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-host-run-multus-certs\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.806707 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.806200 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-multus-conf-dir\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.806707 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.806231 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1ade0d05-b097-449c-8207-734ba764bf02-device-dir\") pod \"aws-ebs-csi-driver-node-djwkt\" (UID: \"1ade0d05-b097-449c-8207-734ba764bf02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djwkt" Jul 16 16:50:50.806707 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.806255 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a686a394-6813-4075-8118-9c99e4c189f2-host\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.806707 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.806284 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/32c71c9a-59a2-4743-8305-520f13688212-host-slash\") pod \"iptables-alerter-6wmn9\" (UID: \"32c71c9a-59a2-4743-8305-520f13688212\") " pod="openshift-network-operator/iptables-alerter-6wmn9" Jul 16 16:50:50.806707 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.806332 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-host-slash\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.806707 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.806340 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cfbe9f0b-b759-4064-9332-3b48c547c003-ovnkube-script-lib\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.806707 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.806362 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cfbe9f0b-b759-4064-9332-3b48c547c003-ovn-node-metrics-cert\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.806707 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.806379 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-log-socket\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.806707 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.806409 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9c75f9a1-e0ed-45af-b4c4-e639989e6d3b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-slv5t\" (UID: \"9c75f9a1-e0ed-45af-b4c4-e639989e6d3b\") " pod="openshift-multus/multus-additional-cni-plugins-slv5t" Jul 16 16:50:50.807390 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.806445 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-host-slash\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.807390 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.806461 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1dd02f2a-342f-46a3-bbba-b630378fa3db-serviceca\") pod \"node-ca-ln8q5\" (UID: \"1dd02f2a-342f-46a3-bbba-b630378fa3db\") " pod="openshift-image-registry/node-ca-ln8q5" Jul 16 16:50:50.807390 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.806512 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-multus-conf-dir\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.807390 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.806523 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a686a394-6813-4075-8118-9c99e4c189f2-var-lib-kubelet\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.807390 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.806652 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldnbr\" (UniqueName: \"kubernetes.io/projected/a686a394-6813-4075-8118-9c99e4c189f2-kube-api-access-ldnbr\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.807390 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.806682 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmdck\" (UniqueName: \"kubernetes.io/projected/f7e16013-94ca-4a61-b551-f3f3b66d3cb1-kube-api-access-wmdck\") pod \"node-resolver-fttnf\" (UID: \"f7e16013-94ca-4a61-b551-f3f3b66d3cb1\") " pod="openshift-dns/node-resolver-fttnf" Jul 16 16:50:50.807390 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.806752 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9c75f9a1-e0ed-45af-b4c4-e639989e6d3b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-slv5t\" (UID: \"9c75f9a1-e0ed-45af-b4c4-e639989e6d3b\") " pod="openshift-multus/multus-additional-cni-plugins-slv5t" Jul 16 16:50:50.807390 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.806782 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-multus-socket-dir-parent\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.807390 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.806811 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a686a394-6813-4075-8118-9c99e4c189f2-etc-sysconfig\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.807390 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.806793 2571 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 16 16:50:50.807390 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.806911 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-multus-socket-dir-parent\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.807390 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.806946 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a686a394-6813-4075-8118-9c99e4c189f2-etc-sysctl-conf\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.807390 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.806977 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/598a7966-d69c-4867-9936-667f0c47ed23-dbus\") pod \"global-pull-secret-syncer-wd65t\" (UID: \"598a7966-d69c-4867-9936-667f0c47ed23\") " pod="kube-system/global-pull-secret-syncer-wd65t" Jul 16 16:50:50.807390 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.806946 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9c75f9a1-e0ed-45af-b4c4-e639989e6d3b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-slv5t\" (UID: \"9c75f9a1-e0ed-45af-b4c4-e639989e6d3b\") " pod="openshift-multus/multus-additional-cni-plugins-slv5t" Jul 16 16:50:50.807390 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.807055 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pwkp\" (UniqueName: \"kubernetes.io/projected/cfbe9f0b-b759-4064-9332-3b48c547c003-kube-api-access-6pwkp\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.807390 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.807081 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-host-run-netns\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.807390 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.807124 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4k8kc\" (UniqueName: \"kubernetes.io/projected/32c71c9a-59a2-4743-8305-520f13688212-kube-api-access-4k8kc\") pod \"iptables-alerter-6wmn9\" (UID: \"32c71c9a-59a2-4743-8305-520f13688212\") " pod="openshift-network-operator/iptables-alerter-6wmn9" Jul 16 16:50:50.808039 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.807153 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-run-systemd\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.808039 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.807321 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.808039 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.807348 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9c75f9a1-e0ed-45af-b4c4-e639989e6d3b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-slv5t\" (UID: \"9c75f9a1-e0ed-45af-b4c4-e639989e6d3b\") " pod="openshift-multus/multus-additional-cni-plugins-slv5t" Jul 16 16:50:50.808039 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.807385 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-hostroot\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.808039 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.807411 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-run-systemd\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.808039 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.807429 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjcwr\" (UniqueName: \"kubernetes.io/projected/1dd02f2a-342f-46a3-bbba-b630378fa3db-kube-api-access-mjcwr\") pod \"node-ca-ln8q5\" (UID: \"1dd02f2a-342f-46a3-bbba-b630378fa3db\") " pod="openshift-image-registry/node-ca-ln8q5" Jul 16 16:50:50.808039 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.807501 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx468\" (UniqueName: \"kubernetes.io/projected/1c20e16c-0039-4705-a02f-d2a8a5c66ae1-kube-api-access-hx468\") pod \"network-check-target-zfpzk\" (UID: \"1c20e16c-0039-4705-a02f-d2a8a5c66ae1\") " pod="openshift-network-diagnostics/network-check-target-zfpzk" Jul 16 16:50:50.808039 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.807562 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a686a394-6813-4075-8118-9c99e4c189f2-etc-tuned\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.808039 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.807583 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/27c7185c-f0e1-4486-882a-bf9275296975-konnectivity-ca\") pod \"konnectivity-agent-df9xm\" (UID: \"27c7185c-f0e1-4486-882a-bf9275296975\") " pod="kube-system/konnectivity-agent-df9xm" Jul 16 16:50:50.808039 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.807612 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cfbe9f0b-b759-4064-9332-3b48c547c003-ovnkube-config\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.808039 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.807648 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-host-var-lib-cni-multus\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.808039 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.807674 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-host-run-netns\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.808039 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.807712 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1ade0d05-b097-449c-8207-734ba764bf02-sys-fs\") pod \"aws-ebs-csi-driver-node-djwkt\" (UID: \"1ade0d05-b097-449c-8207-734ba764bf02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djwkt" Jul 16 16:50:50.808039 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.807806 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cfbe9f0b-b759-4064-9332-3b48c547c003-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.808039 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.807859 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-hostroot\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.808700 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.808153 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a686a394-6813-4075-8118-9c99e4c189f2-sys\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.808700 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.808166 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b7428d07-679c-4e1c-bbaf-36230bfb797b-host-var-lib-cni-multus\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.808700 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.808412 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cfbe9f0b-b759-4064-9332-3b48c547c003-ovnkube-config\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.810273 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.810254 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cfbe9f0b-b759-4064-9332-3b48c547c003-ovn-node-metrics-cert\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.816118 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.816096 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvjmf\" (UniqueName: \"kubernetes.io/projected/2341bdcc-01b6-426c-8189-71be7fdc2e8b-kube-api-access-dvjmf\") pod \"network-metrics-daemon-4p6nc\" (UID: \"2341bdcc-01b6-426c-8189-71be7fdc2e8b\") " pod="openshift-multus/network-metrics-daemon-4p6nc" Jul 16 16:50:50.816356 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.816338 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ht7hx\" (UniqueName: \"kubernetes.io/projected/b7428d07-679c-4e1c-bbaf-36230bfb797b-kube-api-access-ht7hx\") pod \"multus-zn2rb\" (UID: \"b7428d07-679c-4e1c-bbaf-36230bfb797b\") " pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.816407 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.816387 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pwkp\" (UniqueName: \"kubernetes.io/projected/cfbe9f0b-b759-4064-9332-3b48c547c003-kube-api-access-6pwkp\") pod \"ovnkube-node-qd7bg\" (UID: \"cfbe9f0b-b759-4064-9332-3b48c547c003\") " pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.817029 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.817014 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sk8f\" (UniqueName: \"kubernetes.io/projected/9c75f9a1-e0ed-45af-b4c4-e639989e6d3b-kube-api-access-9sk8f\") pod \"multus-additional-cni-plugins-slv5t\" (UID: \"9c75f9a1-e0ed-45af-b4c4-e639989e6d3b\") " pod="openshift-multus/multus-additional-cni-plugins-slv5t" Jul 16 16:50:50.909236 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909215 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a686a394-6813-4075-8118-9c99e4c189f2-tmp\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.909318 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909238 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a686a394-6813-4075-8118-9c99e4c189f2-etc-modprobe-d\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.909318 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909260 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7e16013-94ca-4a61-b551-f3f3b66d3cb1-tmp-dir\") pod \"node-resolver-fttnf\" (UID: \"f7e16013-94ca-4a61-b551-f3f3b66d3cb1\") " pod="openshift-dns/node-resolver-fttnf" Jul 16 16:50:50.909318 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909275 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1ade0d05-b097-449c-8207-734ba764bf02-registration-dir\") pod \"aws-ebs-csi-driver-node-djwkt\" (UID: \"1ade0d05-b097-449c-8207-734ba764bf02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djwkt" Jul 16 16:50:50.909318 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909288 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a686a394-6813-4075-8118-9c99e4c189f2-etc-kubernetes\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.909318 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909301 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a686a394-6813-4075-8118-9c99e4c189f2-etc-sysctl-d\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.909318 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909313 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a686a394-6813-4075-8118-9c99e4c189f2-lib-modules\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.909584 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909326 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/598a7966-d69c-4867-9936-667f0c47ed23-kubelet-config\") pod \"global-pull-secret-syncer-wd65t\" (UID: \"598a7966-d69c-4867-9936-667f0c47ed23\") " pod="kube-system/global-pull-secret-syncer-wd65t" Jul 16 16:50:50.909584 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909356 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1ade0d05-b097-449c-8207-734ba764bf02-kubelet-dir\") pod \"aws-ebs-csi-driver-node-djwkt\" (UID: \"1ade0d05-b097-449c-8207-734ba764bf02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djwkt" Jul 16 16:50:50.909584 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909372 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a686a394-6813-4075-8118-9c99e4c189f2-etc-modprobe-d\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.909584 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909383 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r9xl\" (UniqueName: \"kubernetes.io/projected/1ade0d05-b097-449c-8207-734ba764bf02-kube-api-access-9r9xl\") pod \"aws-ebs-csi-driver-node-djwkt\" (UID: \"1ade0d05-b097-449c-8207-734ba764bf02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djwkt" Jul 16 16:50:50.909584 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909416 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1dd02f2a-342f-46a3-bbba-b630378fa3db-host\") pod \"node-ca-ln8q5\" (UID: \"1dd02f2a-342f-46a3-bbba-b630378fa3db\") " pod="openshift-image-registry/node-ca-ln8q5" Jul 16 16:50:50.909584 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909409 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1ade0d05-b097-449c-8207-734ba764bf02-registration-dir\") pod \"aws-ebs-csi-driver-node-djwkt\" (UID: \"1ade0d05-b097-449c-8207-734ba764bf02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djwkt" Jul 16 16:50:50.909584 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909443 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a686a394-6813-4075-8118-9c99e4c189f2-etc-systemd\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.909584 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909448 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a686a394-6813-4075-8118-9c99e4c189f2-lib-modules\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.909584 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909443 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/598a7966-d69c-4867-9936-667f0c47ed23-kubelet-config\") pod \"global-pull-secret-syncer-wd65t\" (UID: \"598a7966-d69c-4867-9936-667f0c47ed23\") " pod="kube-system/global-pull-secret-syncer-wd65t" Jul 16 16:50:50.909584 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909452 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a686a394-6813-4075-8118-9c99e4c189f2-etc-sysctl-d\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.909584 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909485 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a686a394-6813-4075-8118-9c99e4c189f2-etc-systemd\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.909584 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909495 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a686a394-6813-4075-8118-9c99e4c189f2-etc-kubernetes\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.909584 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909525 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1ade0d05-b097-449c-8207-734ba764bf02-kubelet-dir\") pod \"aws-ebs-csi-driver-node-djwkt\" (UID: \"1ade0d05-b097-449c-8207-734ba764bf02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djwkt" Jul 16 16:50:50.909584 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909488 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f7e16013-94ca-4a61-b551-f3f3b66d3cb1-hosts-file\") pod \"node-resolver-fttnf\" (UID: \"f7e16013-94ca-4a61-b551-f3f3b66d3cb1\") " pod="openshift-dns/node-resolver-fttnf" Jul 16 16:50:50.909584 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909531 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f7e16013-94ca-4a61-b551-f3f3b66d3cb1-hosts-file\") pod \"node-resolver-fttnf\" (UID: \"f7e16013-94ca-4a61-b551-f3f3b66d3cb1\") " pod="openshift-dns/node-resolver-fttnf" Jul 16 16:50:50.909584 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909536 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7e16013-94ca-4a61-b551-f3f3b66d3cb1-tmp-dir\") pod \"node-resolver-fttnf\" (UID: \"f7e16013-94ca-4a61-b551-f3f3b66d3cb1\") " pod="openshift-dns/node-resolver-fttnf" Jul 16 16:50:50.909584 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909572 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1ade0d05-b097-449c-8207-734ba764bf02-socket-dir\") pod \"aws-ebs-csi-driver-node-djwkt\" (UID: \"1ade0d05-b097-449c-8207-734ba764bf02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djwkt" Jul 16 16:50:50.910387 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909599 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/598a7966-d69c-4867-9936-667f0c47ed23-original-pull-secret\") pod \"global-pull-secret-syncer-wd65t\" (UID: \"598a7966-d69c-4867-9936-667f0c47ed23\") " pod="kube-system/global-pull-secret-syncer-wd65t" Jul 16 16:50:50.910387 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909617 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1dd02f2a-342f-46a3-bbba-b630378fa3db-host\") pod \"node-ca-ln8q5\" (UID: \"1dd02f2a-342f-46a3-bbba-b630378fa3db\") " pod="openshift-image-registry/node-ca-ln8q5" Jul 16 16:50:50.910387 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909627 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1ade0d05-b097-449c-8207-734ba764bf02-etc-selinux\") pod \"aws-ebs-csi-driver-node-djwkt\" (UID: \"1ade0d05-b097-449c-8207-734ba764bf02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djwkt" Jul 16 16:50:50.910387 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909664 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a686a394-6813-4075-8118-9c99e4c189f2-run\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.910387 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909678 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/27c7185c-f0e1-4486-882a-bf9275296975-agent-certs\") pod \"konnectivity-agent-df9xm\" (UID: \"27c7185c-f0e1-4486-882a-bf9275296975\") " pod="kube-system/konnectivity-agent-df9xm" Jul 16 16:50:50.910387 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909701 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/32c71c9a-59a2-4743-8305-520f13688212-iptables-alerter-script\") pod \"iptables-alerter-6wmn9\" (UID: \"32c71c9a-59a2-4743-8305-520f13688212\") " pod="openshift-network-operator/iptables-alerter-6wmn9" Jul 16 16:50:50.910387 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:50.909707 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 16:50:50.910387 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909754 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a686a394-6813-4075-8118-9c99e4c189f2-run\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.910387 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909752 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1ade0d05-b097-449c-8207-734ba764bf02-device-dir\") pod \"aws-ebs-csi-driver-node-djwkt\" (UID: \"1ade0d05-b097-449c-8207-734ba764bf02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djwkt" Jul 16 16:50:50.910387 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909699 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1ade0d05-b097-449c-8207-734ba764bf02-socket-dir\") pod \"aws-ebs-csi-driver-node-djwkt\" (UID: \"1ade0d05-b097-449c-8207-734ba764bf02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djwkt" Jul 16 16:50:50.910387 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:50.909793 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/598a7966-d69c-4867-9936-667f0c47ed23-original-pull-secret podName:598a7966-d69c-4867-9936-667f0c47ed23 nodeName:}" failed. No retries permitted until 2026-07-16 16:50:51.409775291 +0000 UTC m=+3.151350569 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/598a7966-d69c-4867-9936-667f0c47ed23-original-pull-secret") pod "global-pull-secret-syncer-wd65t" (UID: "598a7966-d69c-4867-9936-667f0c47ed23") : object "kube-system"/"original-pull-secret" not registered Jul 16 16:50:50.910387 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909800 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1ade0d05-b097-449c-8207-734ba764bf02-etc-selinux\") pod \"aws-ebs-csi-driver-node-djwkt\" (UID: \"1ade0d05-b097-449c-8207-734ba764bf02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djwkt" Jul 16 16:50:50.910387 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909816 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a686a394-6813-4075-8118-9c99e4c189f2-host\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.910387 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909846 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/32c71c9a-59a2-4743-8305-520f13688212-host-slash\") pod \"iptables-alerter-6wmn9\" (UID: \"32c71c9a-59a2-4743-8305-520f13688212\") " pod="openshift-network-operator/iptables-alerter-6wmn9" Jul 16 16:50:50.910387 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909857 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a686a394-6813-4075-8118-9c99e4c189f2-host\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.910387 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909872 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1dd02f2a-342f-46a3-bbba-b630378fa3db-serviceca\") pod \"node-ca-ln8q5\" (UID: \"1dd02f2a-342f-46a3-bbba-b630378fa3db\") " pod="openshift-image-registry/node-ca-ln8q5" Jul 16 16:50:50.910387 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909897 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a686a394-6813-4075-8118-9c99e4c189f2-var-lib-kubelet\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.910956 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909906 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/32c71c9a-59a2-4743-8305-520f13688212-host-slash\") pod \"iptables-alerter-6wmn9\" (UID: \"32c71c9a-59a2-4743-8305-520f13688212\") " pod="openshift-network-operator/iptables-alerter-6wmn9" Jul 16 16:50:50.910956 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909923 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldnbr\" (UniqueName: \"kubernetes.io/projected/a686a394-6813-4075-8118-9c99e4c189f2-kube-api-access-ldnbr\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.910956 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909955 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmdck\" (UniqueName: \"kubernetes.io/projected/f7e16013-94ca-4a61-b551-f3f3b66d3cb1-kube-api-access-wmdck\") pod \"node-resolver-fttnf\" (UID: \"f7e16013-94ca-4a61-b551-f3f3b66d3cb1\") " pod="openshift-dns/node-resolver-fttnf" Jul 16 16:50:50.910956 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909983 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a686a394-6813-4075-8118-9c99e4c189f2-var-lib-kubelet\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.910956 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.909983 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a686a394-6813-4075-8118-9c99e4c189f2-etc-sysconfig\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.910956 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.910038 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a686a394-6813-4075-8118-9c99e4c189f2-etc-sysctl-conf\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.910956 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.910060 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/598a7966-d69c-4867-9936-667f0c47ed23-dbus\") pod \"global-pull-secret-syncer-wd65t\" (UID: \"598a7966-d69c-4867-9936-667f0c47ed23\") " pod="kube-system/global-pull-secret-syncer-wd65t" Jul 16 16:50:50.910956 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.910091 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4k8kc\" (UniqueName: \"kubernetes.io/projected/32c71c9a-59a2-4743-8305-520f13688212-kube-api-access-4k8kc\") pod \"iptables-alerter-6wmn9\" (UID: \"32c71c9a-59a2-4743-8305-520f13688212\") " pod="openshift-network-operator/iptables-alerter-6wmn9" Jul 16 16:50:50.910956 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.910145 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjcwr\" (UniqueName: \"kubernetes.io/projected/1dd02f2a-342f-46a3-bbba-b630378fa3db-kube-api-access-mjcwr\") pod \"node-ca-ln8q5\" (UID: \"1dd02f2a-342f-46a3-bbba-b630378fa3db\") " pod="openshift-image-registry/node-ca-ln8q5" Jul 16 16:50:50.910956 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.910177 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx468\" (UniqueName: \"kubernetes.io/projected/1c20e16c-0039-4705-a02f-d2a8a5c66ae1-kube-api-access-hx468\") pod \"network-check-target-zfpzk\" (UID: \"1c20e16c-0039-4705-a02f-d2a8a5c66ae1\") " pod="openshift-network-diagnostics/network-check-target-zfpzk" Jul 16 16:50:50.910956 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.910199 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a686a394-6813-4075-8118-9c99e4c189f2-etc-tuned\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.910956 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.910223 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/27c7185c-f0e1-4486-882a-bf9275296975-konnectivity-ca\") pod \"konnectivity-agent-df9xm\" (UID: \"27c7185c-f0e1-4486-882a-bf9275296975\") " pod="kube-system/konnectivity-agent-df9xm" Jul 16 16:50:50.910956 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.910252 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a686a394-6813-4075-8118-9c99e4c189f2-etc-sysctl-conf\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.910956 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.910257 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1ade0d05-b097-449c-8207-734ba764bf02-sys-fs\") pod \"aws-ebs-csi-driver-node-djwkt\" (UID: \"1ade0d05-b097-449c-8207-734ba764bf02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djwkt" Jul 16 16:50:50.910956 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.910283 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/32c71c9a-59a2-4743-8305-520f13688212-iptables-alerter-script\") pod \"iptables-alerter-6wmn9\" (UID: \"32c71c9a-59a2-4743-8305-520f13688212\") " pod="openshift-network-operator/iptables-alerter-6wmn9" Jul 16 16:50:50.910956 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.910298 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a686a394-6813-4075-8118-9c99e4c189f2-sys\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.910956 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.910310 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1ade0d05-b097-449c-8207-734ba764bf02-sys-fs\") pod \"aws-ebs-csi-driver-node-djwkt\" (UID: \"1ade0d05-b097-449c-8207-734ba764bf02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djwkt" Jul 16 16:50:50.911525 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.910339 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1ade0d05-b097-449c-8207-734ba764bf02-device-dir\") pod \"aws-ebs-csi-driver-node-djwkt\" (UID: \"1ade0d05-b097-449c-8207-734ba764bf02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djwkt" Jul 16 16:50:50.911525 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.910344 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1dd02f2a-342f-46a3-bbba-b630378fa3db-serviceca\") pod \"node-ca-ln8q5\" (UID: \"1dd02f2a-342f-46a3-bbba-b630378fa3db\") " pod="openshift-image-registry/node-ca-ln8q5" Jul 16 16:50:50.911525 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.910406 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a686a394-6813-4075-8118-9c99e4c189f2-sys\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.911525 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.910406 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a686a394-6813-4075-8118-9c99e4c189f2-etc-sysconfig\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.911525 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.910407 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/598a7966-d69c-4867-9936-667f0c47ed23-dbus\") pod \"global-pull-secret-syncer-wd65t\" (UID: \"598a7966-d69c-4867-9936-667f0c47ed23\") " pod="kube-system/global-pull-secret-syncer-wd65t" Jul 16 16:50:50.911525 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.910926 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/27c7185c-f0e1-4486-882a-bf9275296975-konnectivity-ca\") pod \"konnectivity-agent-df9xm\" (UID: \"27c7185c-f0e1-4486-882a-bf9275296975\") " pod="kube-system/konnectivity-agent-df9xm" Jul 16 16:50:50.911525 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.911330 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a686a394-6813-4075-8118-9c99e4c189f2-tmp\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.911933 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.911915 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/27c7185c-f0e1-4486-882a-bf9275296975-agent-certs\") pod \"konnectivity-agent-df9xm\" (UID: \"27c7185c-f0e1-4486-882a-bf9275296975\") " pod="kube-system/konnectivity-agent-df9xm" Jul 16 16:50:50.912605 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.912590 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a686a394-6813-4075-8118-9c99e4c189f2-etc-tuned\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.917707 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:50.917690 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 16:50:50.917707 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:50.917707 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 16:50:50.917850 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:50.917716 2571 projected.go:196] Error preparing data for projected volume kube-api-access-hx468 for pod openshift-network-diagnostics/network-check-target-zfpzk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 16:50:50.917850 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:50.917779 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c20e16c-0039-4705-a02f-d2a8a5c66ae1-kube-api-access-hx468 podName:1c20e16c-0039-4705-a02f-d2a8a5c66ae1 nodeName:}" failed. No retries permitted until 2026-07-16 16:50:51.417757844 +0000 UTC m=+3.159333123 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-hx468" (UniqueName: "kubernetes.io/projected/1c20e16c-0039-4705-a02f-d2a8a5c66ae1-kube-api-access-hx468") pod "network-check-target-zfpzk" (UID: "1c20e16c-0039-4705-a02f-d2a8a5c66ae1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 16:50:50.919592 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.919565 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r9xl\" (UniqueName: \"kubernetes.io/projected/1ade0d05-b097-449c-8207-734ba764bf02-kube-api-access-9r9xl\") pod \"aws-ebs-csi-driver-node-djwkt\" (UID: \"1ade0d05-b097-449c-8207-734ba764bf02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djwkt" Jul 16 16:50:50.919672 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.919594 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldnbr\" (UniqueName: \"kubernetes.io/projected/a686a394-6813-4075-8118-9c99e4c189f2-kube-api-access-ldnbr\") pod \"tuned-dgn58\" (UID: \"a686a394-6813-4075-8118-9c99e4c189f2\") " pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:50.919672 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.919609 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4k8kc\" (UniqueName: \"kubernetes.io/projected/32c71c9a-59a2-4743-8305-520f13688212-kube-api-access-4k8kc\") pod \"iptables-alerter-6wmn9\" (UID: \"32c71c9a-59a2-4743-8305-520f13688212\") " pod="openshift-network-operator/iptables-alerter-6wmn9" Jul 16 16:50:50.921048 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.921029 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmdck\" (UniqueName: \"kubernetes.io/projected/f7e16013-94ca-4a61-b551-f3f3b66d3cb1-kube-api-access-wmdck\") pod \"node-resolver-fttnf\" (UID: \"f7e16013-94ca-4a61-b551-f3f3b66d3cb1\") " pod="openshift-dns/node-resolver-fttnf" Jul 16 16:50:50.921208 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.921190 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjcwr\" (UniqueName: \"kubernetes.io/projected/1dd02f2a-342f-46a3-bbba-b630378fa3db-kube-api-access-mjcwr\") pod \"node-ca-ln8q5\" (UID: \"1dd02f2a-342f-46a3-bbba-b630378fa3db\") " pod="openshift-image-registry/node-ca-ln8q5" Jul 16 16:50:50.969500 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.969456 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:50:50.978605 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:50.978587 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfbe9f0b_b759_4064_9332_3b48c547c003.slice/crio-46416b2043ecd7f1d51e55e498f4721362253275569034fcbf67d01182f1ed55 WatchSource:0}: Error finding container 46416b2043ecd7f1d51e55e498f4721362253275569034fcbf67d01182f1ed55: Status 404 returned error can't find the container with id 46416b2043ecd7f1d51e55e498f4721362253275569034fcbf67d01182f1ed55 Jul 16 16:50:50.979244 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:50.979230 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zn2rb" Jul 16 16:50:50.987071 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:50.987050 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7428d07_679c_4e1c_bbaf_36230bfb797b.slice/crio-d7e1536b6d4a70da5375a15b01f60b00961a818a88ce0619bd202acb70d68250 WatchSource:0}: Error finding container d7e1536b6d4a70da5375a15b01f60b00961a818a88ce0619bd202acb70d68250: Status 404 returned error can't find the container with id d7e1536b6d4a70da5375a15b01f60b00961a818a88ce0619bd202acb70d68250 Jul 16 16:50:51.005861 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:51.005843 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-slv5t" Jul 16 16:50:51.012678 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:51.012665 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dgn58" Jul 16 16:50:51.013400 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:51.013386 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c75f9a1_e0ed_45af_b4c4_e639989e6d3b.slice/crio-c2d954297a06fbaa6fc8440ae7730ee35d9b40a128def35165a1a5b3cc1930e5 WatchSource:0}: Error finding container c2d954297a06fbaa6fc8440ae7730ee35d9b40a128def35165a1a5b3cc1930e5: Status 404 returned error can't find the container with id c2d954297a06fbaa6fc8440ae7730ee35d9b40a128def35165a1a5b3cc1930e5 Jul 16 16:50:51.035463 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:51.035445 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fttnf" Jul 16 16:50:51.036549 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:51.036533 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-df9xm" Jul 16 16:50:51.039950 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:51.039935 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djwkt" Jul 16 16:50:51.043774 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:51.043753 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7e16013_94ca_4a61_b551_f3f3b66d3cb1.slice/crio-b6f28c7be91e3bead1bc299a286aed7f028969025cfbc73873b2b5178235f333 WatchSource:0}: Error finding container b6f28c7be91e3bead1bc299a286aed7f028969025cfbc73873b2b5178235f333: Status 404 returned error can't find the container with id b6f28c7be91e3bead1bc299a286aed7f028969025cfbc73873b2b5178235f333 Jul 16 16:50:51.044386 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:51.044368 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6wmn9" Jul 16 16:50:51.047255 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:51.047236 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27c7185c_f0e1_4486_882a_bf9275296975.slice/crio-5ae62e61a9a6308361d46d096bccc3692f057239cdd88425f1b8b962ca411a1b WatchSource:0}: Error finding container 5ae62e61a9a6308361d46d096bccc3692f057239cdd88425f1b8b962ca411a1b: Status 404 returned error can't find the container with id 5ae62e61a9a6308361d46d096bccc3692f057239cdd88425f1b8b962ca411a1b Jul 16 16:50:51.048289 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:51.048270 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ade0d05_b097_449c_8207_734ba764bf02.slice/crio-7c29d9c0f4181b7b0baf06f16beaa2fe5d6ff19f4a281024780f75b800666939 WatchSource:0}: Error finding container 7c29d9c0f4181b7b0baf06f16beaa2fe5d6ff19f4a281024780f75b800666939: Status 404 returned error can't find the container with id 7c29d9c0f4181b7b0baf06f16beaa2fe5d6ff19f4a281024780f75b800666939 Jul 16 16:50:51.048924 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:51.048909 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ln8q5" Jul 16 16:50:51.054299 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:51.054284 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32c71c9a_59a2_4743_8305_520f13688212.slice/crio-9eded99cd3c7cec40e57108195081c780c652bbd45151fc1e24925ab4ad3abdf WatchSource:0}: Error finding container 9eded99cd3c7cec40e57108195081c780c652bbd45151fc1e24925ab4ad3abdf: Status 404 returned error can't find the container with id 9eded99cd3c7cec40e57108195081c780c652bbd45151fc1e24925ab4ad3abdf Jul 16 16:50:51.058348 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:50:51.058330 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dd02f2a_342f_46a3_bbba_b630378fa3db.slice/crio-03e3c75d283652dbd97154fb5ac75891dba37bd9dd763a0463099850066db280 WatchSource:0}: Error finding container 03e3c75d283652dbd97154fb5ac75891dba37bd9dd763a0463099850066db280: Status 404 returned error can't find the container with id 03e3c75d283652dbd97154fb5ac75891dba37bd9dd763a0463099850066db280 Jul 16 16:50:51.313711 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:51.313660 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2341bdcc-01b6-426c-8189-71be7fdc2e8b-metrics-certs\") pod \"network-metrics-daemon-4p6nc\" (UID: \"2341bdcc-01b6-426c-8189-71be7fdc2e8b\") " pod="openshift-multus/network-metrics-daemon-4p6nc" Jul 16 16:50:51.313811 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:51.313799 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 16:50:51.313853 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:51.313849 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2341bdcc-01b6-426c-8189-71be7fdc2e8b-metrics-certs podName:2341bdcc-01b6-426c-8189-71be7fdc2e8b nodeName:}" failed. No retries permitted until 2026-07-16 16:50:52.313829285 +0000 UTC m=+4.055404559 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2341bdcc-01b6-426c-8189-71be7fdc2e8b-metrics-certs") pod "network-metrics-daemon-4p6nc" (UID: "2341bdcc-01b6-426c-8189-71be7fdc2e8b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 16:50:51.414115 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:51.414088 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/598a7966-d69c-4867-9936-667f0c47ed23-original-pull-secret\") pod \"global-pull-secret-syncer-wd65t\" (UID: \"598a7966-d69c-4867-9936-667f0c47ed23\") " pod="kube-system/global-pull-secret-syncer-wd65t" Jul 16 16:50:51.414299 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:51.414223 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 16:50:51.414299 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:51.414270 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/598a7966-d69c-4867-9936-667f0c47ed23-original-pull-secret podName:598a7966-d69c-4867-9936-667f0c47ed23 nodeName:}" failed. No retries permitted until 2026-07-16 16:50:52.414254078 +0000 UTC m=+4.155829354 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/598a7966-d69c-4867-9936-667f0c47ed23-original-pull-secret") pod "global-pull-secret-syncer-wd65t" (UID: "598a7966-d69c-4867-9936-667f0c47ed23") : object "kube-system"/"original-pull-secret" not registered Jul 16 16:50:51.515158 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:51.515128 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx468\" (UniqueName: \"kubernetes.io/projected/1c20e16c-0039-4705-a02f-d2a8a5c66ae1-kube-api-access-hx468\") pod \"network-check-target-zfpzk\" (UID: \"1c20e16c-0039-4705-a02f-d2a8a5c66ae1\") " pod="openshift-network-diagnostics/network-check-target-zfpzk" Jul 16 16:50:51.515416 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:51.515281 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 16:50:51.515416 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:51.515304 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 16:50:51.515416 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:51.515316 2571 projected.go:196] Error preparing data for projected volume kube-api-access-hx468 for pod openshift-network-diagnostics/network-check-target-zfpzk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 16:50:51.515416 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:51.515366 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c20e16c-0039-4705-a02f-d2a8a5c66ae1-kube-api-access-hx468 podName:1c20e16c-0039-4705-a02f-d2a8a5c66ae1 nodeName:}" failed. No retries permitted until 2026-07-16 16:50:52.515348467 +0000 UTC m=+4.256923758 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-hx468" (UniqueName: "kubernetes.io/projected/1c20e16c-0039-4705-a02f-d2a8a5c66ae1-kube-api-access-hx468") pod "network-check-target-zfpzk" (UID: "1c20e16c-0039-4705-a02f-d2a8a5c66ae1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 16:50:51.751519 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:51.751440 2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 16:45:50 +0000 UTC" deadline="2028-02-21 18:41:57.798823308 +0000 UTC" Jul 16 16:50:51.751519 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:51.751476 2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14041h51m6.047352211s" Jul 16 16:50:51.784349 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:51.784231 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ln8q5" event={"ID":"1dd02f2a-342f-46a3-bbba-b630378fa3db","Type":"ContainerStarted","Data":"03e3c75d283652dbd97154fb5ac75891dba37bd9dd763a0463099850066db280"} Jul 16 16:50:51.808670 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:51.808570 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-df9xm" event={"ID":"27c7185c-f0e1-4486-882a-bf9275296975","Type":"ContainerStarted","Data":"5ae62e61a9a6308361d46d096bccc3692f057239cdd88425f1b8b962ca411a1b"} Jul 16 16:50:51.817877 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:51.817846 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fttnf" event={"ID":"f7e16013-94ca-4a61-b551-f3f3b66d3cb1","Type":"ContainerStarted","Data":"b6f28c7be91e3bead1bc299a286aed7f028969025cfbc73873b2b5178235f333"} Jul 16 16:50:51.825551 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:51.825490 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dgn58" event={"ID":"a686a394-6813-4075-8118-9c99e4c189f2","Type":"ContainerStarted","Data":"1ecfa8f2d00864dd47718bf6809c2d2d03383980fca08aa275dd02d34f2ad7a2"} Jul 16 16:50:51.830906 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:51.830605 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-slv5t" event={"ID":"9c75f9a1-e0ed-45af-b4c4-e639989e6d3b","Type":"ContainerStarted","Data":"c2d954297a06fbaa6fc8440ae7730ee35d9b40a128def35165a1a5b3cc1930e5"} Jul 16 16:50:51.837308 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:51.837250 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6wmn9" event={"ID":"32c71c9a-59a2-4743-8305-520f13688212","Type":"ContainerStarted","Data":"9eded99cd3c7cec40e57108195081c780c652bbd45151fc1e24925ab4ad3abdf"} Jul 16 16:50:51.841326 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:51.841300 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djwkt" event={"ID":"1ade0d05-b097-449c-8207-734ba764bf02","Type":"ContainerStarted","Data":"7c29d9c0f4181b7b0baf06f16beaa2fe5d6ff19f4a281024780f75b800666939"} Jul 16 16:50:51.842777 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:51.842754 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zn2rb" event={"ID":"b7428d07-679c-4e1c-bbaf-36230bfb797b","Type":"ContainerStarted","Data":"d7e1536b6d4a70da5375a15b01f60b00961a818a88ce0619bd202acb70d68250"} Jul 16 16:50:51.849767 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:51.849742 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" event={"ID":"cfbe9f0b-b759-4064-9332-3b48c547c003","Type":"ContainerStarted","Data":"46416b2043ecd7f1d51e55e498f4721362253275569034fcbf67d01182f1ed55"} Jul 16 16:50:52.320623 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:52.320594 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2341bdcc-01b6-426c-8189-71be7fdc2e8b-metrics-certs\") pod \"network-metrics-daemon-4p6nc\" (UID: \"2341bdcc-01b6-426c-8189-71be7fdc2e8b\") " pod="openshift-multus/network-metrics-daemon-4p6nc" Jul 16 16:50:52.320780 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:52.320764 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 16:50:52.320849 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:52.320824 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2341bdcc-01b6-426c-8189-71be7fdc2e8b-metrics-certs podName:2341bdcc-01b6-426c-8189-71be7fdc2e8b nodeName:}" failed. No retries permitted until 2026-07-16 16:50:54.320805313 +0000 UTC m=+6.062380603 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2341bdcc-01b6-426c-8189-71be7fdc2e8b-metrics-certs") pod "network-metrics-daemon-4p6nc" (UID: "2341bdcc-01b6-426c-8189-71be7fdc2e8b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 16:50:52.421885 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:52.421854 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/598a7966-d69c-4867-9936-667f0c47ed23-original-pull-secret\") pod \"global-pull-secret-syncer-wd65t\" (UID: \"598a7966-d69c-4867-9936-667f0c47ed23\") " pod="kube-system/global-pull-secret-syncer-wd65t" Jul 16 16:50:52.422023 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:52.422008 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 16:50:52.422080 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:52.422071 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/598a7966-d69c-4867-9936-667f0c47ed23-original-pull-secret podName:598a7966-d69c-4867-9936-667f0c47ed23 nodeName:}" failed. No retries permitted until 2026-07-16 16:50:54.422053314 +0000 UTC m=+6.163628589 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/598a7966-d69c-4867-9936-667f0c47ed23-original-pull-secret") pod "global-pull-secret-syncer-wd65t" (UID: "598a7966-d69c-4867-9936-667f0c47ed23") : object "kube-system"/"original-pull-secret" not registered Jul 16 16:50:52.523107 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:52.523035 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx468\" (UniqueName: \"kubernetes.io/projected/1c20e16c-0039-4705-a02f-d2a8a5c66ae1-kube-api-access-hx468\") pod \"network-check-target-zfpzk\" (UID: \"1c20e16c-0039-4705-a02f-d2a8a5c66ae1\") " pod="openshift-network-diagnostics/network-check-target-zfpzk" Jul 16 16:50:52.523228 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:52.523171 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 16:50:52.523228 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:52.523187 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 16:50:52.523228 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:52.523199 2571 projected.go:196] Error preparing data for projected volume kube-api-access-hx468 for pod openshift-network-diagnostics/network-check-target-zfpzk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 16:50:52.523377 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:52.523274 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c20e16c-0039-4705-a02f-d2a8a5c66ae1-kube-api-access-hx468 podName:1c20e16c-0039-4705-a02f-d2a8a5c66ae1 nodeName:}" failed. No retries permitted until 2026-07-16 16:50:54.523255648 +0000 UTC m=+6.264830934 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-hx468" (UniqueName: "kubernetes.io/projected/1c20e16c-0039-4705-a02f-d2a8a5c66ae1-kube-api-access-hx468") pod "network-check-target-zfpzk" (UID: "1c20e16c-0039-4705-a02f-d2a8a5c66ae1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 16:50:52.751936 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:52.751873 2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 16:45:50 +0000 UTC" deadline="2028-02-26 08:19:23.793013156 +0000 UTC" Jul 16 16:50:52.751936 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:52.751928 2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14151h28m31.041089257s" Jul 16 16:50:52.771857 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:52.771834 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4p6nc" Jul 16 16:50:52.771994 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:52.771958 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4p6nc" podUID="2341bdcc-01b6-426c-8189-71be7fdc2e8b" Jul 16 16:50:52.772371 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:52.772351 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zfpzk" Jul 16 16:50:52.772557 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:52.772537 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zfpzk" podUID="1c20e16c-0039-4705-a02f-d2a8a5c66ae1" Jul 16 16:50:52.772976 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:52.772820 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wd65t" Jul 16 16:50:52.772976 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:52.772910 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wd65t" podUID="598a7966-d69c-4867-9936-667f0c47ed23" Jul 16 16:50:52.861849 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:52.861747 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal" event={"ID":"b70782e48d7de5f190e2cd3c95d4336b","Type":"ContainerStarted","Data":"47e1c1bd67f045eece68935b26cf0a4a5fc07fa928f546e29435edd7ca73c3ee"} Jul 16 16:50:54.338936 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:54.338834 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2341bdcc-01b6-426c-8189-71be7fdc2e8b-metrics-certs\") pod \"network-metrics-daemon-4p6nc\" (UID: \"2341bdcc-01b6-426c-8189-71be7fdc2e8b\") " pod="openshift-multus/network-metrics-daemon-4p6nc" Jul 16 16:50:54.339320 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:54.339004 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 16:50:54.339320 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:54.339073 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2341bdcc-01b6-426c-8189-71be7fdc2e8b-metrics-certs podName:2341bdcc-01b6-426c-8189-71be7fdc2e8b nodeName:}" failed. No retries permitted until 2026-07-16 16:50:58.339051471 +0000 UTC m=+10.080626752 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2341bdcc-01b6-426c-8189-71be7fdc2e8b-metrics-certs") pod "network-metrics-daemon-4p6nc" (UID: "2341bdcc-01b6-426c-8189-71be7fdc2e8b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 16:50:54.439796 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:54.439764 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/598a7966-d69c-4867-9936-667f0c47ed23-original-pull-secret\") pod \"global-pull-secret-syncer-wd65t\" (UID: \"598a7966-d69c-4867-9936-667f0c47ed23\") " pod="kube-system/global-pull-secret-syncer-wd65t" Jul 16 16:50:54.439944 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:54.439920 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 16:50:54.440014 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:54.439980 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/598a7966-d69c-4867-9936-667f0c47ed23-original-pull-secret podName:598a7966-d69c-4867-9936-667f0c47ed23 nodeName:}" failed. No retries permitted until 2026-07-16 16:50:58.439962752 +0000 UTC m=+10.181538031 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/598a7966-d69c-4867-9936-667f0c47ed23-original-pull-secret") pod "global-pull-secret-syncer-wd65t" (UID: "598a7966-d69c-4867-9936-667f0c47ed23") : object "kube-system"/"original-pull-secret" not registered Jul 16 16:50:54.540768 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:54.540721 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx468\" (UniqueName: \"kubernetes.io/projected/1c20e16c-0039-4705-a02f-d2a8a5c66ae1-kube-api-access-hx468\") pod \"network-check-target-zfpzk\" (UID: \"1c20e16c-0039-4705-a02f-d2a8a5c66ae1\") " pod="openshift-network-diagnostics/network-check-target-zfpzk" Jul 16 16:50:54.540960 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:54.540945 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 16:50:54.541023 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:54.540972 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 16:50:54.541023 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:54.540984 2571 projected.go:196] Error preparing data for projected volume kube-api-access-hx468 for pod openshift-network-diagnostics/network-check-target-zfpzk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 16:50:54.541089 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:54.541033 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c20e16c-0039-4705-a02f-d2a8a5c66ae1-kube-api-access-hx468 podName:1c20e16c-0039-4705-a02f-d2a8a5c66ae1 nodeName:}" failed. No retries permitted until 2026-07-16 16:50:58.54101529 +0000 UTC m=+10.282590568 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-hx468" (UniqueName: "kubernetes.io/projected/1c20e16c-0039-4705-a02f-d2a8a5c66ae1-kube-api-access-hx468") pod "network-check-target-zfpzk" (UID: "1c20e16c-0039-4705-a02f-d2a8a5c66ae1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 16:50:54.772706 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:54.772636 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4p6nc" Jul 16 16:50:54.772862 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:54.772633 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wd65t" Jul 16 16:50:54.772862 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:54.772631 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zfpzk" Jul 16 16:50:54.772862 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:54.772815 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4p6nc" podUID="2341bdcc-01b6-426c-8189-71be7fdc2e8b" Jul 16 16:50:54.773025 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:54.772935 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zfpzk" podUID="1c20e16c-0039-4705-a02f-d2a8a5c66ae1" Jul 16 16:50:54.773025 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:54.773012 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wd65t" podUID="598a7966-d69c-4867-9936-667f0c47ed23" Jul 16 16:50:56.772272 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:56.771846 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zfpzk" Jul 16 16:50:56.772272 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:56.771845 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4p6nc" Jul 16 16:50:56.772272 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:56.771984 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zfpzk" podUID="1c20e16c-0039-4705-a02f-d2a8a5c66ae1" Jul 16 16:50:56.772272 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:56.772005 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wd65t" Jul 16 16:50:56.772272 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:56.772132 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4p6nc" podUID="2341bdcc-01b6-426c-8189-71be7fdc2e8b" Jul 16 16:50:56.772272 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:56.772232 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wd65t" podUID="598a7966-d69c-4867-9936-667f0c47ed23" Jul 16 16:50:58.375561 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:58.375526 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2341bdcc-01b6-426c-8189-71be7fdc2e8b-metrics-certs\") pod \"network-metrics-daemon-4p6nc\" (UID: \"2341bdcc-01b6-426c-8189-71be7fdc2e8b\") " pod="openshift-multus/network-metrics-daemon-4p6nc" Jul 16 16:50:58.376064 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:58.375712 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 16:50:58.376064 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:58.375796 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2341bdcc-01b6-426c-8189-71be7fdc2e8b-metrics-certs podName:2341bdcc-01b6-426c-8189-71be7fdc2e8b nodeName:}" failed. No retries permitted until 2026-07-16 16:51:06.375774908 +0000 UTC m=+18.117350199 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2341bdcc-01b6-426c-8189-71be7fdc2e8b-metrics-certs") pod "network-metrics-daemon-4p6nc" (UID: "2341bdcc-01b6-426c-8189-71be7fdc2e8b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 16:50:58.476948 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:58.476860 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/598a7966-d69c-4867-9936-667f0c47ed23-original-pull-secret\") pod \"global-pull-secret-syncer-wd65t\" (UID: \"598a7966-d69c-4867-9936-667f0c47ed23\") " pod="kube-system/global-pull-secret-syncer-wd65t" Jul 16 16:50:58.477109 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:58.476971 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 16:50:58.477109 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:58.477034 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/598a7966-d69c-4867-9936-667f0c47ed23-original-pull-secret podName:598a7966-d69c-4867-9936-667f0c47ed23 nodeName:}" failed. No retries permitted until 2026-07-16 16:51:06.477016127 +0000 UTC m=+18.218591401 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/598a7966-d69c-4867-9936-667f0c47ed23-original-pull-secret") pod "global-pull-secret-syncer-wd65t" (UID: "598a7966-d69c-4867-9936-667f0c47ed23") : object "kube-system"/"original-pull-secret" not registered Jul 16 16:50:58.578387 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:58.578259 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx468\" (UniqueName: \"kubernetes.io/projected/1c20e16c-0039-4705-a02f-d2a8a5c66ae1-kube-api-access-hx468\") pod \"network-check-target-zfpzk\" (UID: \"1c20e16c-0039-4705-a02f-d2a8a5c66ae1\") " pod="openshift-network-diagnostics/network-check-target-zfpzk" Jul 16 16:50:58.578552 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:58.578412 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 16:50:58.578552 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:58.578440 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 16:50:58.578552 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:58.578452 2571 projected.go:196] Error preparing data for projected volume kube-api-access-hx468 for pod openshift-network-diagnostics/network-check-target-zfpzk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 16:50:58.578552 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:58.578511 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c20e16c-0039-4705-a02f-d2a8a5c66ae1-kube-api-access-hx468 podName:1c20e16c-0039-4705-a02f-d2a8a5c66ae1 nodeName:}" failed. No retries permitted until 2026-07-16 16:51:06.578491393 +0000 UTC m=+18.320066671 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-hx468" (UniqueName: "kubernetes.io/projected/1c20e16c-0039-4705-a02f-d2a8a5c66ae1-kube-api-access-hx468") pod "network-check-target-zfpzk" (UID: "1c20e16c-0039-4705-a02f-d2a8a5c66ae1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 16:50:58.772786 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:58.772704 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4p6nc" Jul 16 16:50:58.773543 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:58.773007 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4p6nc" podUID="2341bdcc-01b6-426c-8189-71be7fdc2e8b" Jul 16 16:50:58.773543 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:58.773366 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zfpzk" Jul 16 16:50:58.773543 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:58.773449 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zfpzk" podUID="1c20e16c-0039-4705-a02f-d2a8a5c66ae1" Jul 16 16:50:58.773781 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:50:58.773587 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wd65t" Jul 16 16:50:58.773781 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:50:58.773698 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wd65t" podUID="598a7966-d69c-4867-9936-667f0c47ed23" Jul 16 16:51:00.771369 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:00.771340 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zfpzk" Jul 16 16:51:00.771369 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:00.771358 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wd65t" Jul 16 16:51:00.771879 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:00.771340 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4p6nc" Jul 16 16:51:00.771879 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:00.771441 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zfpzk" podUID="1c20e16c-0039-4705-a02f-d2a8a5c66ae1" Jul 16 16:51:00.771879 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:00.771562 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4p6nc" podUID="2341bdcc-01b6-426c-8189-71be7fdc2e8b" Jul 16 16:51:00.771879 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:00.771700 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wd65t" podUID="598a7966-d69c-4867-9936-667f0c47ed23" Jul 16 16:51:02.771184 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:02.771114 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4p6nc" Jul 16 16:51:02.771184 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:02.771131 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zfpzk" Jul 16 16:51:02.771629 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:02.771102 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wd65t" Jul 16 16:51:02.771629 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:02.771239 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4p6nc" podUID="2341bdcc-01b6-426c-8189-71be7fdc2e8b" Jul 16 16:51:02.771629 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:02.771313 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wd65t" podUID="598a7966-d69c-4867-9936-667f0c47ed23" Jul 16 16:51:02.771629 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:02.771393 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zfpzk" podUID="1c20e16c-0039-4705-a02f-d2a8a5c66ae1" Jul 16 16:51:04.771702 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:04.771525 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4p6nc" Jul 16 16:51:04.772113 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:04.771525 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zfpzk" Jul 16 16:51:04.772113 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:04.771553 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wd65t" Jul 16 16:51:04.772113 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:04.771815 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4p6nc" podUID="2341bdcc-01b6-426c-8189-71be7fdc2e8b" Jul 16 16:51:04.772504 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:04.772388 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zfpzk" podUID="1c20e16c-0039-4705-a02f-d2a8a5c66ae1" Jul 16 16:51:04.773211 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:04.773172 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wd65t" podUID="598a7966-d69c-4867-9936-667f0c47ed23" Jul 16 16:51:06.441471 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:06.441334 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2341bdcc-01b6-426c-8189-71be7fdc2e8b-metrics-certs\") pod \"network-metrics-daemon-4p6nc\" (UID: \"2341bdcc-01b6-426c-8189-71be7fdc2e8b\") " pod="openshift-multus/network-metrics-daemon-4p6nc" Jul 16 16:51:06.441916 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:06.441496 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 16:51:06.441916 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:06.441566 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2341bdcc-01b6-426c-8189-71be7fdc2e8b-metrics-certs podName:2341bdcc-01b6-426c-8189-71be7fdc2e8b nodeName:}" failed. No retries permitted until 2026-07-16 16:51:22.441546778 +0000 UTC m=+34.183122070 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2341bdcc-01b6-426c-8189-71be7fdc2e8b-metrics-certs") pod "network-metrics-daemon-4p6nc" (UID: "2341bdcc-01b6-426c-8189-71be7fdc2e8b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 16:51:06.542559 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:06.542529 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/598a7966-d69c-4867-9936-667f0c47ed23-original-pull-secret\") pod \"global-pull-secret-syncer-wd65t\" (UID: \"598a7966-d69c-4867-9936-667f0c47ed23\") " pod="kube-system/global-pull-secret-syncer-wd65t" Jul 16 16:51:06.542676 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:06.542651 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 16:51:06.542747 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:06.542709 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/598a7966-d69c-4867-9936-667f0c47ed23-original-pull-secret podName:598a7966-d69c-4867-9936-667f0c47ed23 nodeName:}" failed. No retries permitted until 2026-07-16 16:51:22.542690966 +0000 UTC m=+34.284266242 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/598a7966-d69c-4867-9936-667f0c47ed23-original-pull-secret") pod "global-pull-secret-syncer-wd65t" (UID: "598a7966-d69c-4867-9936-667f0c47ed23") : object "kube-system"/"original-pull-secret" not registered Jul 16 16:51:06.642904 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:06.642874 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx468\" (UniqueName: \"kubernetes.io/projected/1c20e16c-0039-4705-a02f-d2a8a5c66ae1-kube-api-access-hx468\") pod \"network-check-target-zfpzk\" (UID: \"1c20e16c-0039-4705-a02f-d2a8a5c66ae1\") " pod="openshift-network-diagnostics/network-check-target-zfpzk" Jul 16 16:51:06.643056 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:06.643042 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 16:51:06.643122 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:06.643063 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 16:51:06.643122 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:06.643074 2571 projected.go:196] Error preparing data for projected volume kube-api-access-hx468 for pod openshift-network-diagnostics/network-check-target-zfpzk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 16:51:06.643213 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:06.643135 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c20e16c-0039-4705-a02f-d2a8a5c66ae1-kube-api-access-hx468 podName:1c20e16c-0039-4705-a02f-d2a8a5c66ae1 nodeName:}" failed. No retries permitted until 2026-07-16 16:51:22.643114684 +0000 UTC m=+34.384689976 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-hx468" (UniqueName: "kubernetes.io/projected/1c20e16c-0039-4705-a02f-d2a8a5c66ae1-kube-api-access-hx468") pod "network-check-target-zfpzk" (UID: "1c20e16c-0039-4705-a02f-d2a8a5c66ae1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 16:51:06.771012 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:06.770948 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4p6nc" Jul 16 16:51:06.771012 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:06.770970 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wd65t" Jul 16 16:51:06.771182 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:06.770948 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zfpzk" Jul 16 16:51:06.771182 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:06.771062 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4p6nc" podUID="2341bdcc-01b6-426c-8189-71be7fdc2e8b" Jul 16 16:51:06.771182 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:06.771152 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zfpzk" podUID="1c20e16c-0039-4705-a02f-d2a8a5c66ae1" Jul 16 16:51:06.771296 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:06.771224 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wd65t" podUID="598a7966-d69c-4867-9936-667f0c47ed23" Jul 16 16:51:08.772068 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:08.772041 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4p6nc" Jul 16 16:51:08.772487 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:08.772148 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4p6nc" podUID="2341bdcc-01b6-426c-8189-71be7fdc2e8b" Jul 16 16:51:08.772487 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:08.772235 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zfpzk" Jul 16 16:51:08.772487 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:08.772351 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zfpzk" podUID="1c20e16c-0039-4705-a02f-d2a8a5c66ae1" Jul 16 16:51:08.772487 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:08.772396 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wd65t" Jul 16 16:51:08.772487 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:08.772462 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wd65t" podUID="598a7966-d69c-4867-9936-667f0c47ed23" Jul 16 16:51:10.772085 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:10.772062 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4p6nc" Jul 16 16:51:10.772452 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:10.772175 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4p6nc" podUID="2341bdcc-01b6-426c-8189-71be7fdc2e8b" Jul 16 16:51:10.772452 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:10.772062 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wd65t" Jul 16 16:51:10.772452 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:10.772266 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wd65t" podUID="598a7966-d69c-4867-9936-667f0c47ed23" Jul 16 16:51:10.772452 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:10.772286 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zfpzk" Jul 16 16:51:10.772452 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:10.772345 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zfpzk" podUID="1c20e16c-0039-4705-a02f-d2a8a5c66ae1" Jul 16 16:51:10.905245 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:10.905200 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.ec2.internal" event={"ID":"683869ec7e8dd47873bd79d88a644cf6","Type":"ContainerStarted","Data":"3e0e2c13d04757c56596a88aa1a360e49eb35cf4e4da8ff817cd0905b8874ee5"} Jul 16 16:51:10.906508 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:10.906487 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dgn58" event={"ID":"a686a394-6813-4075-8118-9c99e4c189f2","Type":"ContainerStarted","Data":"27f775ddb5ff2b07ac8dbba5e3453dc13ae77c3b7657e4abcda0641e94cdb255"} Jul 16 16:51:10.907854 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:10.907833 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zn2rb" event={"ID":"b7428d07-679c-4e1c-bbaf-36230bfb797b","Type":"ContainerStarted","Data":"fd6952f8158b57aef3387532e49981779b442366f3cdcc86103d7d55c8e0846d"} Jul 16 16:51:10.909144 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:10.909122 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" event={"ID":"cfbe9f0b-b759-4064-9332-3b48c547c003","Type":"ContainerStarted","Data":"a6d98bbedb4ca789b289e381e202c8a5400ae78fa16094e3f7c37042f4a90339"} Jul 16 16:51:10.929829 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:10.929793 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.ec2.internal" podStartSLOduration=21.929783114 podStartE2EDuration="21.929783114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 16:50:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:51:10.928856792 +0000 UTC m=+22.670432089" watchObservedRunningTime="2026-07-16 16:51:10.929783114 +0000 UTC m=+22.671358409" Jul 16 16:51:10.949430 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:10.949388 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-zn2rb" podStartSLOduration=3.153782247 podStartE2EDuration="22.949363798s" totalImagesPullingTime="19.795581551s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 16:50:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:51:10.949061168 +0000 UTC m=+22.690636463" watchObservedRunningTime="2026-07-16 16:51:10.949363798 +0000 UTC m=+22.690939093" Jul 16 16:51:10.969750 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:10.969554 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-dgn58" podStartSLOduration=3.262100605 podStartE2EDuration="22.969537367s" totalImagesPullingTime="19.707436762s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 16:50:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:51:10.968758111 +0000 UTC m=+22.710333406" watchObservedRunningTime="2026-07-16 16:51:10.969537367 +0000 UTC m=+22.711112668" Jul 16 16:51:11.914610 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:11.914329 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qd7bg_cfbe9f0b-b759-4064-9332-3b48c547c003/ovn-acl-logging/0.log" Jul 16 16:51:11.915369 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:11.915010 2571 generic.go:356] "Generic (PLEG): container finished" podID="cfbe9f0b-b759-4064-9332-3b48c547c003" containerID="023f2b36b0b8d86abd747a531e2d1213f161f9ecc7c18a2c97d9b7c763eb2a33" exitCode=1 Jul 16 16:51:11.915369 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:11.915048 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" event={"ID":"cfbe9f0b-b759-4064-9332-3b48c547c003","Type":"ContainerStarted","Data":"96b5bfdd1a520ada7e8d9c4129fe4543ff0e6454b79305af1f79d775b7a93884"} Jul 16 16:51:11.915369 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:11.915166 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" event={"ID":"cfbe9f0b-b759-4064-9332-3b48c547c003","Type":"ContainerStarted","Data":"e3b501d3560b7c740e2c02dcf15cabd2491709ae754e17e7774b337eb5375b3e"} Jul 16 16:51:11.915369 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:11.915219 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" event={"ID":"cfbe9f0b-b759-4064-9332-3b48c547c003","Type":"ContainerStarted","Data":"e6434ca7c1eb6dfdd82c00dcbfd4f8873e730cbb12b8ebd4cb5bc5b12a374684"} Jul 16 16:51:11.915369 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:11.915267 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" event={"ID":"cfbe9f0b-b759-4064-9332-3b48c547c003","Type":"ContainerStarted","Data":"8c427f3a318f4f44518c47bc6282212cde182c07587f5f6d9b860a67dcf2c9d9"} Jul 16 16:51:11.915369 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:11.915314 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" event={"ID":"cfbe9f0b-b759-4064-9332-3b48c547c003","Type":"ContainerDied","Data":"023f2b36b0b8d86abd747a531e2d1213f161f9ecc7c18a2c97d9b7c763eb2a33"} Jul 16 16:51:11.916847 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:11.916824 2571 generic.go:356] "Generic (PLEG): container finished" podID="b70782e48d7de5f190e2cd3c95d4336b" containerID="ab19fb892ad141801e920838672e9463ed33f4c794168226a260bddf64a505bb" exitCode=0 Jul 16 16:51:11.916967 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:11.916896 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal" event={"ID":"b70782e48d7de5f190e2cd3c95d4336b","Type":"ContainerDied","Data":"ab19fb892ad141801e920838672e9463ed33f4c794168226a260bddf64a505bb"} Jul 16 16:51:11.919574 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:11.919177 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ln8q5" event={"ID":"1dd02f2a-342f-46a3-bbba-b630378fa3db","Type":"ContainerStarted","Data":"0b0bd28a42964ecddb6eccf80341461c5fd10e1056ed11722214d3ff61bb06f2"} Jul 16 16:51:11.922950 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:11.922930 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-df9xm" event={"ID":"27c7185c-f0e1-4486-882a-bf9275296975","Type":"ContainerStarted","Data":"2b007872b5daf330ffe8aa238105dca2d8f6ae1aaaad13ccbcfa2b4fbac1c1d6"} Jul 16 16:51:11.924379 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:11.924357 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fttnf" event={"ID":"f7e16013-94ca-4a61-b551-f3f3b66d3cb1","Type":"ContainerStarted","Data":"1f50bd1400d36776f45091ea4d9da4279e1215ece9e39308b01a278398ae1951"} Jul 16 16:51:11.926077 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:11.926051 2571 generic.go:356] "Generic (PLEG): container finished" podID="9c75f9a1-e0ed-45af-b4c4-e639989e6d3b" containerID="ede23d2dfae316985c08066a9b659b0a460adbed4ddb3dfc508ce0982491a5ef" exitCode=0 Jul 16 16:51:11.926187 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:11.926101 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-slv5t" event={"ID":"9c75f9a1-e0ed-45af-b4c4-e639989e6d3b","Type":"ContainerDied","Data":"ede23d2dfae316985c08066a9b659b0a460adbed4ddb3dfc508ce0982491a5ef"} Jul 16 16:51:11.928047 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:11.928022 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djwkt" event={"ID":"1ade0d05-b097-449c-8207-734ba764bf02","Type":"ContainerStarted","Data":"269cc96190ea85f0e03a8480ef78adbaaffde0ff43ef6f52911cbf707e3f91ef"} Jul 16 16:51:11.952249 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:11.952204 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-df9xm" podStartSLOduration=3.270439505 podStartE2EDuration="22.95218792s" totalImagesPullingTime="19.681748415s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:50:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:51:11.951616088 +0000 UTC m=+23.693191384" watchObservedRunningTime="2026-07-16 16:51:11.95218792 +0000 UTC m=+23.693763217" Jul 16 16:51:11.967470 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:11.967430 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-ln8q5" podStartSLOduration=3.31226726 podStartE2EDuration="22.967414339s" totalImagesPullingTime="19.655147079s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 16:50:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:51:11.967032348 +0000 UTC m=+23.708607646" watchObservedRunningTime="2026-07-16 16:51:11.967414339 +0000 UTC m=+23.708989637" Jul 16 16:51:11.985525 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:11.983607 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-fttnf" podStartSLOduration=8.049221487 podStartE2EDuration="22.983591827s" totalImagesPullingTime="14.93437034s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 16:50:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:51:11.983079795 +0000 UTC m=+23.724655287" watchObservedRunningTime="2026-07-16 16:51:11.983591827 +0000 UTC m=+23.725167124" Jul 16 16:51:12.746231 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:12.746209 2571 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 16 16:51:12.771320 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:12.771287 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4p6nc" Jul 16 16:51:12.771419 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:12.771297 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wd65t" Jul 16 16:51:12.771481 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:12.771415 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4p6nc" podUID="2341bdcc-01b6-426c-8189-71be7fdc2e8b" Jul 16 16:51:12.771535 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:12.771488 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wd65t" podUID="598a7966-d69c-4867-9936-667f0c47ed23" Jul 16 16:51:12.771535 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:12.771297 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zfpzk" Jul 16 16:51:12.771632 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:12.771575 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zfpzk" podUID="1c20e16c-0039-4705-a02f-d2a8a5c66ae1" Jul 16 16:51:12.772339 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:12.772250 2571 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-16T16:51:12.746228097Z","UUID":"2e05c92e-4b3e-4a1d-aacc-fe4ac0e694d3","Handler":null,"Name":"","Endpoint":""} Jul 16 16:51:12.774172 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:12.774153 2571 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 16 16:51:12.774252 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:12.774191 2571 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 16 16:51:12.932582 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:12.932504 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6wmn9" event={"ID":"32c71c9a-59a2-4743-8305-520f13688212","Type":"ContainerStarted","Data":"5fe9bdfe366c914021a6219b3ba8e1476504d11dc0013158e2efc80ced095dc3"} Jul 16 16:51:12.934710 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:12.934685 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djwkt" event={"ID":"1ade0d05-b097-449c-8207-734ba764bf02","Type":"ContainerStarted","Data":"62b00eb26cb316057d4127ab819247a8cf419038840d8a7dfe44d1341f94c96b"} Jul 16 16:51:12.936583 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:12.936543 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal" event={"ID":"b70782e48d7de5f190e2cd3c95d4336b","Type":"ContainerStarted","Data":"a50f06bb66c75a931cb35a50a0ec330744edbe4487993e3d96581de74ba2678d"} Jul 16 16:51:12.949008 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:12.948966 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-6wmn9" podStartSLOduration=4.274752666 podStartE2EDuration="23.948951139s" totalImagesPullingTime="19.674198473s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 16:50:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:51:12.94866167 +0000 UTC m=+24.690236967" watchObservedRunningTime="2026-07-16 16:51:12.948951139 +0000 UTC m=+24.690526434" Jul 16 16:51:12.965894 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:12.965849 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal" podStartSLOduration=23.965832732 podStartE2EDuration="23.965832732s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 16:50:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:51:12.96556182 +0000 UTC m=+24.707137116" watchObservedRunningTime="2026-07-16 16:51:12.965832732 +0000 UTC m=+24.707408029" Jul 16 16:51:13.942421 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:13.942360 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qd7bg_cfbe9f0b-b759-4064-9332-3b48c547c003/ovn-acl-logging/0.log" Jul 16 16:51:13.942874 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:13.942777 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" event={"ID":"cfbe9f0b-b759-4064-9332-3b48c547c003","Type":"ContainerStarted","Data":"a77456fea836bee8d713b8c2c7b84d1da6dd68ef71b32363d50c63805a90ae63"} Jul 16 16:51:14.772174 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:14.771957 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4p6nc" Jul 16 16:51:14.772439 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:14.771957 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zfpzk" Jul 16 16:51:14.772439 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:14.772069 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wd65t" Jul 16 16:51:14.772439 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:14.772266 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4p6nc" podUID="2341bdcc-01b6-426c-8189-71be7fdc2e8b" Jul 16 16:51:14.772439 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:14.772416 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wd65t" podUID="598a7966-d69c-4867-9936-667f0c47ed23" Jul 16 16:51:14.772637 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:14.772494 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zfpzk" podUID="1c20e16c-0039-4705-a02f-d2a8a5c66ae1" Jul 16 16:51:14.947030 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:14.946996 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djwkt" event={"ID":"1ade0d05-b097-449c-8207-734ba764bf02","Type":"ContainerStarted","Data":"bd67bed88a86dc548e7f6214b2a93873da5c159c74d84acb881843be6ad17cee"} Jul 16 16:51:14.966985 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:14.966928 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djwkt" podStartSLOduration=3.945802518 podStartE2EDuration="25.966912508s" totalImagesPullingTime="22.02110999s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 16:50:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:51:14.966832473 +0000 UTC m=+26.708407781" watchObservedRunningTime="2026-07-16 16:51:14.966912508 +0000 UTC m=+26.708487803" Jul 16 16:51:16.771269 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:16.771079 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zfpzk" Jul 16 16:51:16.771888 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:16.771079 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4p6nc" Jul 16 16:51:16.771888 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:16.771355 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zfpzk" podUID="1c20e16c-0039-4705-a02f-d2a8a5c66ae1" Jul 16 16:51:16.771888 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:16.771453 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4p6nc" podUID="2341bdcc-01b6-426c-8189-71be7fdc2e8b" Jul 16 16:51:16.771888 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:16.771087 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wd65t" Jul 16 16:51:16.771888 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:16.771576 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wd65t" podUID="598a7966-d69c-4867-9936-667f0c47ed23" Jul 16 16:51:16.916944 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:16.916922 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-df9xm" Jul 16 16:51:16.917657 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:16.917639 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-df9xm" Jul 16 16:51:16.952974 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:16.952946 2571 generic.go:356] "Generic (PLEG): container finished" podID="9c75f9a1-e0ed-45af-b4c4-e639989e6d3b" containerID="6c8a8673cd9d6817c4bf04c488b63fc40ab89f7d7e23265e05196e169ec888fd" exitCode=0 Jul 16 16:51:16.953108 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:16.953033 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-slv5t" event={"ID":"9c75f9a1-e0ed-45af-b4c4-e639989e6d3b","Type":"ContainerDied","Data":"6c8a8673cd9d6817c4bf04c488b63fc40ab89f7d7e23265e05196e169ec888fd"} Jul 16 16:51:16.957143 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:16.957124 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qd7bg_cfbe9f0b-b759-4064-9332-3b48c547c003/ovn-acl-logging/0.log" Jul 16 16:51:16.959482 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:16.957549 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" event={"ID":"cfbe9f0b-b759-4064-9332-3b48c547c003","Type":"ContainerStarted","Data":"829b4ecce2bd7ff1659855284560ab4ddc107de6e47010b15fc35259a2629f64"} Jul 16 16:51:16.962360 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:16.959622 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-df9xm" Jul 16 16:51:16.962360 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:16.959670 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:51:16.962360 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:16.959754 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:51:16.962360 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:16.959856 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-df9xm" Jul 16 16:51:16.962360 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:16.960545 2571 scope.go:117] "RemoveContainer" containerID="023f2b36b0b8d86abd747a531e2d1213f161f9ecc7c18a2c97d9b7c763eb2a33" Jul 16 16:51:16.976609 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:16.976589 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:51:17.962719 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:17.962693 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qd7bg_cfbe9f0b-b759-4064-9332-3b48c547c003/ovn-acl-logging/0.log" Jul 16 16:51:17.963145 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:17.963006 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" event={"ID":"cfbe9f0b-b759-4064-9332-3b48c547c003","Type":"ContainerStarted","Data":"769f647671b8c15f84580de83de35f53a65cea58ed5567af9c40613d1d069682"} Jul 16 16:51:17.963192 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:17.963178 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:51:17.976873 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:17.976852 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:51:17.995533 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:17.995491 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" podStartSLOduration=10.190826136 podStartE2EDuration="29.995478232s" totalImagesPullingTime="19.804652096s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 16:50:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:51:17.993789765 +0000 UTC m=+29.735365061" watchObservedRunningTime="2026-07-16 16:51:17.995478232 +0000 UTC m=+29.737053527" Jul 16 16:51:18.257597 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:18.257528 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zfpzk"] Jul 16 16:51:18.257783 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:18.257649 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zfpzk" Jul 16 16:51:18.257783 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:18.257744 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zfpzk" podUID="1c20e16c-0039-4705-a02f-d2a8a5c66ae1" Jul 16 16:51:18.260162 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:18.260139 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wd65t"] Jul 16 16:51:18.260270 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:18.260234 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wd65t" Jul 16 16:51:18.260341 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:18.260321 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wd65t" podUID="598a7966-d69c-4867-9936-667f0c47ed23" Jul 16 16:51:18.263433 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:18.263408 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4p6nc"] Jul 16 16:51:18.263542 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:18.263508 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4p6nc" Jul 16 16:51:18.263625 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:18.263606 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4p6nc" podUID="2341bdcc-01b6-426c-8189-71be7fdc2e8b" Jul 16 16:51:19.771146 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:19.771104 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4p6nc" Jul 16 16:51:19.771146 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:19.771146 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zfpzk" Jul 16 16:51:19.771595 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:19.771230 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4p6nc" podUID="2341bdcc-01b6-426c-8189-71be7fdc2e8b" Jul 16 16:51:19.771595 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:19.771258 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zfpzk" podUID="1c20e16c-0039-4705-a02f-d2a8a5c66ae1" Jul 16 16:51:19.771595 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:19.771289 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wd65t" Jul 16 16:51:19.771595 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:19.771358 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wd65t" podUID="598a7966-d69c-4867-9936-667f0c47ed23" Jul 16 16:51:20.970437 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:20.970228 2571 generic.go:356] "Generic (PLEG): container finished" podID="9c75f9a1-e0ed-45af-b4c4-e639989e6d3b" containerID="3bf3149612566862541652d8c8e928ca28266ff9bd8171cedf093d1a38048042" exitCode=0 Jul 16 16:51:20.970820 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:20.970298 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-slv5t" event={"ID":"9c75f9a1-e0ed-45af-b4c4-e639989e6d3b","Type":"ContainerDied","Data":"3bf3149612566862541652d8c8e928ca28266ff9bd8171cedf093d1a38048042"} Jul 16 16:51:21.771627 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:21.771601 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zfpzk" Jul 16 16:51:21.771785 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:21.771680 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wd65t" Jul 16 16:51:21.771785 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:21.771685 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zfpzk" podUID="1c20e16c-0039-4705-a02f-d2a8a5c66ae1" Jul 16 16:51:21.771785 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:21.771703 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4p6nc" Jul 16 16:51:21.771883 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:21.771793 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wd65t" podUID="598a7966-d69c-4867-9936-667f0c47ed23" Jul 16 16:51:21.771883 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:21.771870 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4p6nc" podUID="2341bdcc-01b6-426c-8189-71be7fdc2e8b" Jul 16 16:51:22.464219 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:22.464162 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2341bdcc-01b6-426c-8189-71be7fdc2e8b-metrics-certs\") pod \"network-metrics-daemon-4p6nc\" (UID: \"2341bdcc-01b6-426c-8189-71be7fdc2e8b\") " pod="openshift-multus/network-metrics-daemon-4p6nc" Jul 16 16:51:22.464554 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:22.464268 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 16:51:22.464554 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:22.464316 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2341bdcc-01b6-426c-8189-71be7fdc2e8b-metrics-certs podName:2341bdcc-01b6-426c-8189-71be7fdc2e8b nodeName:}" failed. No retries permitted until 2026-07-16 16:51:54.464301765 +0000 UTC m=+66.205877038 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2341bdcc-01b6-426c-8189-71be7fdc2e8b-metrics-certs") pod "network-metrics-daemon-4p6nc" (UID: "2341bdcc-01b6-426c-8189-71be7fdc2e8b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 16:51:22.565332 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:22.565304 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/598a7966-d69c-4867-9936-667f0c47ed23-original-pull-secret\") pod \"global-pull-secret-syncer-wd65t\" (UID: \"598a7966-d69c-4867-9936-667f0c47ed23\") " pod="kube-system/global-pull-secret-syncer-wd65t" Jul 16 16:51:22.565461 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:22.565409 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 16:51:22.565524 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:22.565464 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/598a7966-d69c-4867-9936-667f0c47ed23-original-pull-secret podName:598a7966-d69c-4867-9936-667f0c47ed23 nodeName:}" failed. No retries permitted until 2026-07-16 16:51:54.565449204 +0000 UTC m=+66.307024499 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/598a7966-d69c-4867-9936-667f0c47ed23-original-pull-secret") pod "global-pull-secret-syncer-wd65t" (UID: "598a7966-d69c-4867-9936-667f0c47ed23") : object "kube-system"/"original-pull-secret" not registered Jul 16 16:51:22.665661 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:22.665635 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx468\" (UniqueName: \"kubernetes.io/projected/1c20e16c-0039-4705-a02f-d2a8a5c66ae1-kube-api-access-hx468\") pod \"network-check-target-zfpzk\" (UID: \"1c20e16c-0039-4705-a02f-d2a8a5c66ae1\") " pod="openshift-network-diagnostics/network-check-target-zfpzk" Jul 16 16:51:22.665813 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:22.665796 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 16:51:22.665861 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:22.665819 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 16:51:22.665861 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:22.665832 2571 projected.go:196] Error preparing data for projected volume kube-api-access-hx468 for pod openshift-network-diagnostics/network-check-target-zfpzk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 16:51:22.665927 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:22.665873 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c20e16c-0039-4705-a02f-d2a8a5c66ae1-kube-api-access-hx468 podName:1c20e16c-0039-4705-a02f-d2a8a5c66ae1 nodeName:}" failed. No retries permitted until 2026-07-16 16:51:54.665862331 +0000 UTC m=+66.407437606 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-hx468" (UniqueName: "kubernetes.io/projected/1c20e16c-0039-4705-a02f-d2a8a5c66ae1-kube-api-access-hx468") pod "network-check-target-zfpzk" (UID: "1c20e16c-0039-4705-a02f-d2a8a5c66ae1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 16:51:22.977502 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:22.977472 2571 generic.go:356] "Generic (PLEG): container finished" podID="9c75f9a1-e0ed-45af-b4c4-e639989e6d3b" containerID="b90de4aca0909d5b1e35ca01b524ec5f7c445e4f29002c2e6404975b34d9f412" exitCode=0 Jul 16 16:51:22.977616 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:22.977515 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-slv5t" event={"ID":"9c75f9a1-e0ed-45af-b4c4-e639989e6d3b","Type":"ContainerDied","Data":"b90de4aca0909d5b1e35ca01b524ec5f7c445e4f29002c2e6404975b34d9f412"} Jul 16 16:51:23.561159 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.561132 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeReady" Jul 16 16:51:23.561525 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.561226 2571 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 16 16:51:23.605402 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.605380 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5574f576bc-zfnfv"] Jul 16 16:51:23.607345 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.607329 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" Jul 16 16:51:23.610009 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.609987 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 16 16:51:23.610123 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.610075 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 16 16:51:23.610123 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.610099 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 16 16:51:23.610213 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.610204 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2wxqg\"" Jul 16 16:51:23.616145 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.616121 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 16 16:51:23.627266 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.627243 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2bnf8"] Jul 16 16:51:23.628950 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.628929 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5574f576bc-zfnfv"] Jul 16 16:51:23.629027 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.628965 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-xxcr4"] Jul 16 16:51:23.629086 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.629032 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2bnf8" Jul 16 16:51:23.630661 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.630640 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xxcr4" Jul 16 16:51:23.633860 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.633840 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xtf2c\"" Jul 16 16:51:23.633988 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.633972 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-lnc6v\"" Jul 16 16:51:23.634098 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.634082 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 16 16:51:23.634321 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.634304 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 16 16:51:23.634557 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.634538 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 16 16:51:23.634678 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.634615 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 16 16:51:23.634751 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.634700 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 16 16:51:23.641590 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.641563 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2bnf8"] Jul 16 16:51:23.645225 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.645160 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xxcr4"] Jul 16 16:51:23.676015 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.675999 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x772l\" (UniqueName: \"kubernetes.io/projected/d8306746-2174-40cd-9319-7137be8a4f5e-kube-api-access-x772l\") pod \"ingress-canary-xxcr4\" (UID: \"d8306746-2174-40cd-9319-7137be8a4f5e\") " pod="openshift-ingress-canary/ingress-canary-xxcr4" Jul 16 16:51:23.676104 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.676032 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d8306746-2174-40cd-9319-7137be8a4f5e-cert\") pod \"ingress-canary-xxcr4\" (UID: \"d8306746-2174-40cd-9319-7137be8a4f5e\") " pod="openshift-ingress-canary/ingress-canary-xxcr4" Jul 16 16:51:23.676104 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.676049 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/77f8326b-54be-49e5-a855-647cd2cd05f8-registry-tls\") pod \"image-registry-5574f576bc-zfnfv\" (UID: \"77f8326b-54be-49e5-a855-647cd2cd05f8\") " pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" Jul 16 16:51:23.676104 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.676063 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/77f8326b-54be-49e5-a855-647cd2cd05f8-trusted-ca\") pod \"image-registry-5574f576bc-zfnfv\" (UID: \"77f8326b-54be-49e5-a855-647cd2cd05f8\") " pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" Jul 16 16:51:23.676104 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.676086 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwxx4\" (UniqueName: \"kubernetes.io/projected/77f8326b-54be-49e5-a855-647cd2cd05f8-kube-api-access-hwxx4\") pod \"image-registry-5574f576bc-zfnfv\" (UID: \"77f8326b-54be-49e5-a855-647cd2cd05f8\") " pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" Jul 16 16:51:23.676302 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.676122 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/959268a1-18d8-4f7b-8c63-ba175ba63fe9-config-volume\") pod \"dns-default-2bnf8\" (UID: \"959268a1-18d8-4f7b-8c63-ba175ba63fe9\") " pod="openshift-dns/dns-default-2bnf8" Jul 16 16:51:23.676302 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.676152 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/959268a1-18d8-4f7b-8c63-ba175ba63fe9-metrics-tls\") pod \"dns-default-2bnf8\" (UID: \"959268a1-18d8-4f7b-8c63-ba175ba63fe9\") " pod="openshift-dns/dns-default-2bnf8" Jul 16 16:51:23.676302 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.676198 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/77f8326b-54be-49e5-a855-647cd2cd05f8-registry-certificates\") pod \"image-registry-5574f576bc-zfnfv\" (UID: \"77f8326b-54be-49e5-a855-647cd2cd05f8\") " pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" Jul 16 16:51:23.676302 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.676218 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gqsl\" (UniqueName: \"kubernetes.io/projected/959268a1-18d8-4f7b-8c63-ba175ba63fe9-kube-api-access-9gqsl\") pod \"dns-default-2bnf8\" (UID: \"959268a1-18d8-4f7b-8c63-ba175ba63fe9\") " pod="openshift-dns/dns-default-2bnf8" Jul 16 16:51:23.676302 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.676278 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/77f8326b-54be-49e5-a855-647cd2cd05f8-bound-sa-token\") pod \"image-registry-5574f576bc-zfnfv\" (UID: \"77f8326b-54be-49e5-a855-647cd2cd05f8\") " pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" Jul 16 16:51:23.676526 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.676343 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/77f8326b-54be-49e5-a855-647cd2cd05f8-ca-trust-extracted\") pod \"image-registry-5574f576bc-zfnfv\" (UID: \"77f8326b-54be-49e5-a855-647cd2cd05f8\") " pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" Jul 16 16:51:23.676526 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.676400 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/77f8326b-54be-49e5-a855-647cd2cd05f8-image-registry-private-configuration\") pod \"image-registry-5574f576bc-zfnfv\" (UID: \"77f8326b-54be-49e5-a855-647cd2cd05f8\") " pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" Jul 16 16:51:23.676526 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.676419 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/77f8326b-54be-49e5-a855-647cd2cd05f8-installation-pull-secrets\") pod \"image-registry-5574f576bc-zfnfv\" (UID: \"77f8326b-54be-49e5-a855-647cd2cd05f8\") " pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" Jul 16 16:51:23.676526 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.676435 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/959268a1-18d8-4f7b-8c63-ba175ba63fe9-tmp-dir\") pod \"dns-default-2bnf8\" (UID: \"959268a1-18d8-4f7b-8c63-ba175ba63fe9\") " pod="openshift-dns/dns-default-2bnf8" Jul 16 16:51:23.771482 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.771463 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zfpzk" Jul 16 16:51:23.771482 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.771476 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4p6nc" Jul 16 16:51:23.771692 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.771666 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wd65t" Jul 16 16:51:23.777709 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.777682 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/959268a1-18d8-4f7b-8c63-ba175ba63fe9-tmp-dir\") pod \"dns-default-2bnf8\" (UID: \"959268a1-18d8-4f7b-8c63-ba175ba63fe9\") " pod="openshift-dns/dns-default-2bnf8" Jul 16 16:51:23.778467 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.777705 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 16:51:23.778467 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.778160 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x772l\" (UniqueName: \"kubernetes.io/projected/d8306746-2174-40cd-9319-7137be8a4f5e-kube-api-access-x772l\") pod \"ingress-canary-xxcr4\" (UID: \"d8306746-2174-40cd-9319-7137be8a4f5e\") " pod="openshift-ingress-canary/ingress-canary-xxcr4" Jul 16 16:51:23.778467 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.778186 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 16:51:23.778467 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.778207 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d8306746-2174-40cd-9319-7137be8a4f5e-cert\") pod \"ingress-canary-xxcr4\" (UID: \"d8306746-2174-40cd-9319-7137be8a4f5e\") " pod="openshift-ingress-canary/ingress-canary-xxcr4" Jul 16 16:51:23.778467 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.778232 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/77f8326b-54be-49e5-a855-647cd2cd05f8-registry-tls\") pod \"image-registry-5574f576bc-zfnfv\" (UID: \"77f8326b-54be-49e5-a855-647cd2cd05f8\") " pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" Jul 16 16:51:23.778467 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.778252 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-s2b72\"" Jul 16 16:51:23.778467 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.778134 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-h8r6p\"" Jul 16 16:51:23.778467 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.778255 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/77f8326b-54be-49e5-a855-647cd2cd05f8-trusted-ca\") pod \"image-registry-5574f576bc-zfnfv\" (UID: \"77f8326b-54be-49e5-a855-647cd2cd05f8\") " pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" Jul 16 16:51:23.778467 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.778407 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/959268a1-18d8-4f7b-8c63-ba175ba63fe9-tmp-dir\") pod \"dns-default-2bnf8\" (UID: \"959268a1-18d8-4f7b-8c63-ba175ba63fe9\") " pod="openshift-dns/dns-default-2bnf8" Jul 16 16:51:23.778467 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.778439 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwxx4\" (UniqueName: \"kubernetes.io/projected/77f8326b-54be-49e5-a855-647cd2cd05f8-kube-api-access-hwxx4\") pod \"image-registry-5574f576bc-zfnfv\" (UID: \"77f8326b-54be-49e5-a855-647cd2cd05f8\") " pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" Jul 16 16:51:23.778467 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.778467 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/959268a1-18d8-4f7b-8c63-ba175ba63fe9-config-volume\") pod \"dns-default-2bnf8\" (UID: \"959268a1-18d8-4f7b-8c63-ba175ba63fe9\") " pod="openshift-dns/dns-default-2bnf8" Jul 16 16:51:23.779053 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.778504 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/959268a1-18d8-4f7b-8c63-ba175ba63fe9-metrics-tls\") pod \"dns-default-2bnf8\" (UID: \"959268a1-18d8-4f7b-8c63-ba175ba63fe9\") " pod="openshift-dns/dns-default-2bnf8" Jul 16 16:51:23.779053 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:23.778496 2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 16:51:23.779053 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.778533 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/77f8326b-54be-49e5-a855-647cd2cd05f8-registry-certificates\") pod \"image-registry-5574f576bc-zfnfv\" (UID: \"77f8326b-54be-49e5-a855-647cd2cd05f8\") " pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" Jul 16 16:51:23.779053 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:23.778544 2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5574f576bc-zfnfv: secret "image-registry-tls" not found Jul 16 16:51:23.779053 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.778558 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gqsl\" (UniqueName: \"kubernetes.io/projected/959268a1-18d8-4f7b-8c63-ba175ba63fe9-kube-api-access-9gqsl\") pod \"dns-default-2bnf8\" (UID: \"959268a1-18d8-4f7b-8c63-ba175ba63fe9\") " pod="openshift-dns/dns-default-2bnf8" Jul 16 16:51:23.779053 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.778584 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/77f8326b-54be-49e5-a855-647cd2cd05f8-bound-sa-token\") pod \"image-registry-5574f576bc-zfnfv\" (UID: \"77f8326b-54be-49e5-a855-647cd2cd05f8\") " pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" Jul 16 16:51:23.779053 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:23.778634 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/77f8326b-54be-49e5-a855-647cd2cd05f8-registry-tls podName:77f8326b-54be-49e5-a855-647cd2cd05f8 nodeName:}" failed. No retries permitted until 2026-07-16 16:51:24.278608231 +0000 UTC m=+36.020183515 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/77f8326b-54be-49e5-a855-647cd2cd05f8-registry-tls") pod "image-registry-5574f576bc-zfnfv" (UID: "77f8326b-54be-49e5-a855-647cd2cd05f8") : secret "image-registry-tls" not found Jul 16 16:51:23.779053 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.778659 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 16 16:51:23.779053 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.778671 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/77f8326b-54be-49e5-a855-647cd2cd05f8-ca-trust-extracted\") pod \"image-registry-5574f576bc-zfnfv\" (UID: \"77f8326b-54be-49e5-a855-647cd2cd05f8\") " pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" Jul 16 16:51:23.779053 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.778746 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/77f8326b-54be-49e5-a855-647cd2cd05f8-image-registry-private-configuration\") pod \"image-registry-5574f576bc-zfnfv\" (UID: \"77f8326b-54be-49e5-a855-647cd2cd05f8\") " pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" Jul 16 16:51:23.779053 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.778780 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/77f8326b-54be-49e5-a855-647cd2cd05f8-installation-pull-secrets\") pod \"image-registry-5574f576bc-zfnfv\" (UID: \"77f8326b-54be-49e5-a855-647cd2cd05f8\") " pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" Jul 16 16:51:23.779053 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.778855 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 16 16:51:23.779053 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:23.778967 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 16:51:23.779053 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:23.779045 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/959268a1-18d8-4f7b-8c63-ba175ba63fe9-metrics-tls podName:959268a1-18d8-4f7b-8c63-ba175ba63fe9 nodeName:}" failed. No retries permitted until 2026-07-16 16:51:24.279019568 +0000 UTC m=+36.020594854 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/959268a1-18d8-4f7b-8c63-ba175ba63fe9-metrics-tls") pod "dns-default-2bnf8" (UID: "959268a1-18d8-4f7b-8c63-ba175ba63fe9") : secret "dns-default-metrics-tls" not found Jul 16 16:51:23.779717 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.779326 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/77f8326b-54be-49e5-a855-647cd2cd05f8-ca-trust-extracted\") pod \"image-registry-5574f576bc-zfnfv\" (UID: \"77f8326b-54be-49e5-a855-647cd2cd05f8\") " pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" Jul 16 16:51:23.779717 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.779632 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/959268a1-18d8-4f7b-8c63-ba175ba63fe9-config-volume\") pod \"dns-default-2bnf8\" (UID: \"959268a1-18d8-4f7b-8c63-ba175ba63fe9\") " pod="openshift-dns/dns-default-2bnf8" Jul 16 16:51:23.779856 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.779802 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/77f8326b-54be-49e5-a855-647cd2cd05f8-trusted-ca\") pod \"image-registry-5574f576bc-zfnfv\" (UID: \"77f8326b-54be-49e5-a855-647cd2cd05f8\") " pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" Jul 16 16:51:23.780311 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.780256 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/77f8326b-54be-49e5-a855-647cd2cd05f8-registry-certificates\") pod \"image-registry-5574f576bc-zfnfv\" (UID: \"77f8326b-54be-49e5-a855-647cd2cd05f8\") " pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" Jul 16 16:51:23.783150 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.783125 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/77f8326b-54be-49e5-a855-647cd2cd05f8-installation-pull-secrets\") pod \"image-registry-5574f576bc-zfnfv\" (UID: \"77f8326b-54be-49e5-a855-647cd2cd05f8\") " pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" Jul 16 16:51:23.783230 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.783212 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d8306746-2174-40cd-9319-7137be8a4f5e-cert\") pod \"ingress-canary-xxcr4\" (UID: \"d8306746-2174-40cd-9319-7137be8a4f5e\") " pod="openshift-ingress-canary/ingress-canary-xxcr4" Jul 16 16:51:23.783230 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.783219 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/77f8326b-54be-49e5-a855-647cd2cd05f8-image-registry-private-configuration\") pod \"image-registry-5574f576bc-zfnfv\" (UID: \"77f8326b-54be-49e5-a855-647cd2cd05f8\") " pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" Jul 16 16:51:23.796098 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.796072 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x772l\" (UniqueName: \"kubernetes.io/projected/d8306746-2174-40cd-9319-7137be8a4f5e-kube-api-access-x772l\") pod \"ingress-canary-xxcr4\" (UID: \"d8306746-2174-40cd-9319-7137be8a4f5e\") " pod="openshift-ingress-canary/ingress-canary-xxcr4" Jul 16 16:51:23.796356 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.796339 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/77f8326b-54be-49e5-a855-647cd2cd05f8-bound-sa-token\") pod \"image-registry-5574f576bc-zfnfv\" (UID: \"77f8326b-54be-49e5-a855-647cd2cd05f8\") " pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" Jul 16 16:51:23.797125 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.797105 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gqsl\" (UniqueName: \"kubernetes.io/projected/959268a1-18d8-4f7b-8c63-ba175ba63fe9-kube-api-access-9gqsl\") pod \"dns-default-2bnf8\" (UID: \"959268a1-18d8-4f7b-8c63-ba175ba63fe9\") " pod="openshift-dns/dns-default-2bnf8" Jul 16 16:51:23.797424 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.797401 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwxx4\" (UniqueName: \"kubernetes.io/projected/77f8326b-54be-49e5-a855-647cd2cd05f8-kube-api-access-hwxx4\") pod \"image-registry-5574f576bc-zfnfv\" (UID: \"77f8326b-54be-49e5-a855-647cd2cd05f8\") " pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" Jul 16 16:51:23.943538 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:23.943476 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xxcr4" Jul 16 16:51:24.092124 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:24.091982 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xxcr4"] Jul 16 16:51:24.095758 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:51:24.095714 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8306746_2174_40cd_9319_7137be8a4f5e.slice/crio-a49d758871f50339d01bd452ecf7a19d832462b2577b7d7df3cdade9ebbc7b91 WatchSource:0}: Error finding container a49d758871f50339d01bd452ecf7a19d832462b2577b7d7df3cdade9ebbc7b91: Status 404 returned error can't find the container with id a49d758871f50339d01bd452ecf7a19d832462b2577b7d7df3cdade9ebbc7b91 Jul 16 16:51:24.282001 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:24.281920 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/77f8326b-54be-49e5-a855-647cd2cd05f8-registry-tls\") pod \"image-registry-5574f576bc-zfnfv\" (UID: \"77f8326b-54be-49e5-a855-647cd2cd05f8\") " pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" Jul 16 16:51:24.282001 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:24.281950 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/959268a1-18d8-4f7b-8c63-ba175ba63fe9-metrics-tls\") pod \"dns-default-2bnf8\" (UID: \"959268a1-18d8-4f7b-8c63-ba175ba63fe9\") " pod="openshift-dns/dns-default-2bnf8" Jul 16 16:51:24.282139 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:24.282065 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 16:51:24.282139 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:24.282116 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/959268a1-18d8-4f7b-8c63-ba175ba63fe9-metrics-tls podName:959268a1-18d8-4f7b-8c63-ba175ba63fe9 nodeName:}" failed. No retries permitted until 2026-07-16 16:51:25.282100984 +0000 UTC m=+37.023676258 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/959268a1-18d8-4f7b-8c63-ba175ba63fe9-metrics-tls") pod "dns-default-2bnf8" (UID: "959268a1-18d8-4f7b-8c63-ba175ba63fe9") : secret "dns-default-metrics-tls" not found Jul 16 16:51:24.282202 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:24.282069 2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 16:51:24.282202 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:24.282159 2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5574f576bc-zfnfv: secret "image-registry-tls" not found Jul 16 16:51:24.282261 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:24.282218 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/77f8326b-54be-49e5-a855-647cd2cd05f8-registry-tls podName:77f8326b-54be-49e5-a855-647cd2cd05f8 nodeName:}" failed. No retries permitted until 2026-07-16 16:51:25.282204195 +0000 UTC m=+37.023779486 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/77f8326b-54be-49e5-a855-647cd2cd05f8-registry-tls") pod "image-registry-5574f576bc-zfnfv" (UID: "77f8326b-54be-49e5-a855-647cd2cd05f8") : secret "image-registry-tls" not found Jul 16 16:51:24.983827 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:24.983791 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xxcr4" event={"ID":"d8306746-2174-40cd-9319-7137be8a4f5e","Type":"ContainerStarted","Data":"a49d758871f50339d01bd452ecf7a19d832462b2577b7d7df3cdade9ebbc7b91"} Jul 16 16:51:25.288327 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:25.288264 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/77f8326b-54be-49e5-a855-647cd2cd05f8-registry-tls\") pod \"image-registry-5574f576bc-zfnfv\" (UID: \"77f8326b-54be-49e5-a855-647cd2cd05f8\") " pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" Jul 16 16:51:25.288327 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:25.288293 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/959268a1-18d8-4f7b-8c63-ba175ba63fe9-metrics-tls\") pod \"dns-default-2bnf8\" (UID: \"959268a1-18d8-4f7b-8c63-ba175ba63fe9\") " pod="openshift-dns/dns-default-2bnf8" Jul 16 16:51:25.288556 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:25.288395 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 16:51:25.288556 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:25.288408 2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 16:51:25.288556 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:25.288423 2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5574f576bc-zfnfv: secret "image-registry-tls" not found Jul 16 16:51:25.288556 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:25.288443 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/959268a1-18d8-4f7b-8c63-ba175ba63fe9-metrics-tls podName:959268a1-18d8-4f7b-8c63-ba175ba63fe9 nodeName:}" failed. No retries permitted until 2026-07-16 16:51:27.28843058 +0000 UTC m=+39.030005854 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/959268a1-18d8-4f7b-8c63-ba175ba63fe9-metrics-tls") pod "dns-default-2bnf8" (UID: "959268a1-18d8-4f7b-8c63-ba175ba63fe9") : secret "dns-default-metrics-tls" not found Jul 16 16:51:25.288556 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:25.288468 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/77f8326b-54be-49e5-a855-647cd2cd05f8-registry-tls podName:77f8326b-54be-49e5-a855-647cd2cd05f8 nodeName:}" failed. No retries permitted until 2026-07-16 16:51:27.288459318 +0000 UTC m=+39.030034594 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/77f8326b-54be-49e5-a855-647cd2cd05f8-registry-tls") pod "image-registry-5574f576bc-zfnfv" (UID: "77f8326b-54be-49e5-a855-647cd2cd05f8") : secret "image-registry-tls" not found Jul 16 16:51:26.991197 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:26.991164 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xxcr4" event={"ID":"d8306746-2174-40cd-9319-7137be8a4f5e","Type":"ContainerStarted","Data":"285f7551ef7015c3081a6ec3fbd3e3adeec04f67eecc483521422a4128adbb6a"} Jul 16 16:51:27.009377 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:27.009330 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-xxcr4" podStartSLOduration=1.232254747 podStartE2EDuration="4.009314369s" totalImagesPullingTime="2.777059622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:51:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:51:27.008650454 +0000 UTC m=+38.750225751" watchObservedRunningTime="2026-07-16 16:51:27.009314369 +0000 UTC m=+38.750889665" Jul 16 16:51:27.302624 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:27.302593 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/77f8326b-54be-49e5-a855-647cd2cd05f8-registry-tls\") pod \"image-registry-5574f576bc-zfnfv\" (UID: \"77f8326b-54be-49e5-a855-647cd2cd05f8\") " pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" Jul 16 16:51:27.302624 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:27.302625 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/959268a1-18d8-4f7b-8c63-ba175ba63fe9-metrics-tls\") pod \"dns-default-2bnf8\" (UID: \"959268a1-18d8-4f7b-8c63-ba175ba63fe9\") " pod="openshift-dns/dns-default-2bnf8" Jul 16 16:51:27.302880 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:27.302769 2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 16:51:27.302880 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:27.302791 2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5574f576bc-zfnfv: secret "image-registry-tls" not found Jul 16 16:51:27.302880 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:27.302816 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 16:51:27.302880 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:27.302855 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/77f8326b-54be-49e5-a855-647cd2cd05f8-registry-tls podName:77f8326b-54be-49e5-a855-647cd2cd05f8 nodeName:}" failed. No retries permitted until 2026-07-16 16:51:31.302835852 +0000 UTC m=+43.044411142 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/77f8326b-54be-49e5-a855-647cd2cd05f8-registry-tls") pod "image-registry-5574f576bc-zfnfv" (UID: "77f8326b-54be-49e5-a855-647cd2cd05f8") : secret "image-registry-tls" not found Jul 16 16:51:27.302880 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:27.302870 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/959268a1-18d8-4f7b-8c63-ba175ba63fe9-metrics-tls podName:959268a1-18d8-4f7b-8c63-ba175ba63fe9 nodeName:}" failed. No retries permitted until 2026-07-16 16:51:31.302863487 +0000 UTC m=+43.044438761 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/959268a1-18d8-4f7b-8c63-ba175ba63fe9-metrics-tls") pod "dns-default-2bnf8" (UID: "959268a1-18d8-4f7b-8c63-ba175ba63fe9") : secret "dns-default-metrics-tls" not found Jul 16 16:51:30.003006 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:30.002964 2571 generic.go:356] "Generic (PLEG): container finished" podID="9c75f9a1-e0ed-45af-b4c4-e639989e6d3b" containerID="6472d77ec95b231bc655bee1573e99c0acbd91e6add3ced8633b31237911a13a" exitCode=0 Jul 16 16:51:30.003424 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:30.003057 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-slv5t" event={"ID":"9c75f9a1-e0ed-45af-b4c4-e639989e6d3b","Type":"ContainerDied","Data":"6472d77ec95b231bc655bee1573e99c0acbd91e6add3ced8633b31237911a13a"} Jul 16 16:51:31.008506 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:31.008469 2571 generic.go:356] "Generic (PLEG): container finished" podID="9c75f9a1-e0ed-45af-b4c4-e639989e6d3b" containerID="5ce7c210cb3ed386aad79d5417a1dbea5e0082cb6c26f48b22c20a8928b4406f" exitCode=0 Jul 16 16:51:31.008849 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:31.008512 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-slv5t" event={"ID":"9c75f9a1-e0ed-45af-b4c4-e639989e6d3b","Type":"ContainerDied","Data":"5ce7c210cb3ed386aad79d5417a1dbea5e0082cb6c26f48b22c20a8928b4406f"} Jul 16 16:51:31.337235 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:31.337172 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/77f8326b-54be-49e5-a855-647cd2cd05f8-registry-tls\") pod \"image-registry-5574f576bc-zfnfv\" (UID: \"77f8326b-54be-49e5-a855-647cd2cd05f8\") " pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" Jul 16 16:51:31.337235 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:31.337205 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/959268a1-18d8-4f7b-8c63-ba175ba63fe9-metrics-tls\") pod \"dns-default-2bnf8\" (UID: \"959268a1-18d8-4f7b-8c63-ba175ba63fe9\") " pod="openshift-dns/dns-default-2bnf8" Jul 16 16:51:31.337363 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:31.337318 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 16:51:31.337363 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:31.337362 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/959268a1-18d8-4f7b-8c63-ba175ba63fe9-metrics-tls podName:959268a1-18d8-4f7b-8c63-ba175ba63fe9 nodeName:}" failed. No retries permitted until 2026-07-16 16:51:39.337349542 +0000 UTC m=+51.078924817 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/959268a1-18d8-4f7b-8c63-ba175ba63fe9-metrics-tls") pod "dns-default-2bnf8" (UID: "959268a1-18d8-4f7b-8c63-ba175ba63fe9") : secret "dns-default-metrics-tls" not found Jul 16 16:51:31.337430 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:31.337322 2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 16:51:31.337430 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:31.337413 2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5574f576bc-zfnfv: secret "image-registry-tls" not found Jul 16 16:51:31.337485 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:31.337455 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/77f8326b-54be-49e5-a855-647cd2cd05f8-registry-tls podName:77f8326b-54be-49e5-a855-647cd2cd05f8 nodeName:}" failed. No retries permitted until 2026-07-16 16:51:39.33744333 +0000 UTC m=+51.079018616 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/77f8326b-54be-49e5-a855-647cd2cd05f8-registry-tls") pod "image-registry-5574f576bc-zfnfv" (UID: "77f8326b-54be-49e5-a855-647cd2cd05f8") : secret "image-registry-tls" not found Jul 16 16:51:32.015072 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:32.015039 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-slv5t" event={"ID":"9c75f9a1-e0ed-45af-b4c4-e639989e6d3b","Type":"ContainerStarted","Data":"35b60821b6322dd2b204152b72740f6e8c53ab3edff6633617b083123c54379b"} Jul 16 16:51:32.042335 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:32.042289 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-slv5t" podStartSLOduration=9.212214336 podStartE2EDuration="44.042274591s" totalImagesPullingTime="34.830060255s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 16:50:48 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:51:32.041079917 +0000 UTC m=+43.782655248" watchObservedRunningTime="2026-07-16 16:51:32.042274591 +0000 UTC m=+43.783849887" Jul 16 16:51:34.144253 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.144222 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-ct89n"] Jul 16 16:51:34.177574 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.177544 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-ct89n"] Jul 16 16:51:34.177702 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.177643 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-ct89n" Jul 16 16:51:34.180358 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.180334 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-jchj7\"" Jul 16 16:51:34.180358 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.180334 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 16 16:51:34.180503 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.180409 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 16 16:51:34.244368 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.244345 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5tsmr"] Jul 16 16:51:34.255786 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.255763 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cksk\" (UniqueName: \"kubernetes.io/projected/1896b4d7-17ae-46be-98bc-54a304f42eda-kube-api-access-2cksk\") pod \"volume-data-source-validator-78495597f6-ct89n\" (UID: \"1896b4d7-17ae-46be-98bc-54a304f42eda\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-ct89n" Jul 16 16:51:34.265003 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.264983 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fjsn9"] Jul 16 16:51:34.265132 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.265116 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5tsmr" Jul 16 16:51:34.268385 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.268363 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 16 16:51:34.268484 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.268427 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 16 16:51:34.272343 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.272324 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 16 16:51:34.272454 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.272344 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-2qdtq\"" Jul 16 16:51:34.272806 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.272789 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 16 16:51:34.289038 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.289016 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5tsmr"] Jul 16 16:51:34.289123 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.289045 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fjsn9"] Jul 16 16:51:34.289123 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.289105 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fjsn9" Jul 16 16:51:34.292313 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.292292 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 16 16:51:34.292413 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.292337 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 16 16:51:34.292413 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.292305 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-2bk85\"" Jul 16 16:51:34.292413 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.292304 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 16 16:51:34.292413 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.292410 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 16 16:51:34.350974 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.350958 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5ctvn"] Jul 16 16:51:34.356470 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.356442 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/bd2d4507-465f-4002-8953-9e289841cdbb-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-5tsmr\" (UID: \"bd2d4507-465f-4002-8953-9e289841cdbb\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5tsmr" Jul 16 16:51:34.356548 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.356472 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhn6d\" (UniqueName: \"kubernetes.io/projected/32cd8d7c-6e88-483e-a5d7-f028311b4be3-kube-api-access-nhn6d\") pod \"kube-storage-version-migrator-operator-799cb7f958-fjsn9\" (UID: \"32cd8d7c-6e88-483e-a5d7-f028311b4be3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fjsn9" Jul 16 16:51:34.356548 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.356492 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bd2d4507-465f-4002-8953-9e289841cdbb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-5tsmr\" (UID: \"bd2d4507-465f-4002-8953-9e289841cdbb\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5tsmr" Jul 16 16:51:34.356548 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.356521 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/32cd8d7c-6e88-483e-a5d7-f028311b4be3-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-fjsn9\" (UID: \"32cd8d7c-6e88-483e-a5d7-f028311b4be3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fjsn9" Jul 16 16:51:34.356687 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.356554 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cksk\" (UniqueName: \"kubernetes.io/projected/1896b4d7-17ae-46be-98bc-54a304f42eda-kube-api-access-2cksk\") pod \"volume-data-source-validator-78495597f6-ct89n\" (UID: \"1896b4d7-17ae-46be-98bc-54a304f42eda\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-ct89n" Jul 16 16:51:34.356687 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.356584 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/32cd8d7c-6e88-483e-a5d7-f028311b4be3-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-fjsn9\" (UID: \"32cd8d7c-6e88-483e-a5d7-f028311b4be3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fjsn9" Jul 16 16:51:34.356687 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.356631 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrqx9\" (UniqueName: \"kubernetes.io/projected/bd2d4507-465f-4002-8953-9e289841cdbb-kube-api-access-zrqx9\") pod \"cluster-monitoring-operator-5f7f494dd4-5tsmr\" (UID: \"bd2d4507-465f-4002-8953-9e289841cdbb\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5tsmr" Jul 16 16:51:34.363079 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.363061 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-5t9rp"] Jul 16 16:51:34.363204 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.363186 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5ctvn" Jul 16 16:51:34.366274 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.366250 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 16 16:51:34.366624 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.366605 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 16 16:51:34.366897 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.366881 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-wz9sj\"" Jul 16 16:51:34.366983 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.366947 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 16 16:51:34.373177 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.373159 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-57bc6f48c6-kt2wd"] Jul 16 16:51:34.373293 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.373277 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-5t9rp" Jul 16 16:51:34.374431 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.374387 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cksk\" (UniqueName: \"kubernetes.io/projected/1896b4d7-17ae-46be-98bc-54a304f42eda-kube-api-access-2cksk\") pod \"volume-data-source-validator-78495597f6-ct89n\" (UID: \"1896b4d7-17ae-46be-98bc-54a304f42eda\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-ct89n" Jul 16 16:51:34.375801 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.375780 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 16 16:51:34.375899 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.375807 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-h5xjp\"" Jul 16 16:51:34.376186 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.376171 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 16 16:51:34.376260 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.376221 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 16 16:51:34.376315 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.376299 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 16 16:51:34.382057 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.382041 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 16 16:51:34.389247 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.389227 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-r8cn4"] Jul 16 16:51:34.389339 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.389302 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-57bc6f48c6-kt2wd" Jul 16 16:51:34.391818 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.391752 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 16 16:51:34.391893 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.391849 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 16 16:51:34.392211 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.392188 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 16 16:51:34.392320 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.392300 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 16 16:51:34.392425 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.392302 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 16 16:51:34.392601 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.392586 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 16 16:51:34.392673 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.392601 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-8q47g\"" Jul 16 16:51:34.403462 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.403439 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5ctvn"] Jul 16 16:51:34.403538 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.403469 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-5t9rp"] Jul 16 16:51:34.403538 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.403482 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-57bc6f48c6-kt2wd"] Jul 16 16:51:34.403538 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.403497 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-r8cn4"] Jul 16 16:51:34.403632 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.403544 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-r8cn4" Jul 16 16:51:34.406138 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.406123 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-zqhm5\"" Jul 16 16:51:34.406559 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.406532 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 16 16:51:34.406674 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.406583 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 16 16:51:34.406674 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.406612 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 16 16:51:34.406674 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.406584 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 16 16:51:34.414422 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.414401 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 16 16:51:34.448615 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.448592 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-6zvr8"] Jul 16 16:51:34.456961 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.456942 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d2bdea19-f5df-4297-8304-da19a8cc7a62-stats-auth\") pod \"router-default-57bc6f48c6-kt2wd\" (UID: \"d2bdea19-f5df-4297-8304-da19a8cc7a62\") " pod="openshift-ingress/router-default-57bc6f48c6-kt2wd" Jul 16 16:51:34.457013 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.456966 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d67e5a27-8c54-42e3-af31-f7abfd8ddf95-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-5ctvn\" (UID: \"d67e5a27-8c54-42e3-af31-f7abfd8ddf95\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5ctvn" Jul 16 16:51:34.457013 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.456982 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f637d961-672d-4aa2-a49a-fd021c2d4df2-config\") pod \"console-operator-7866497494-r8cn4\" (UID: \"f637d961-672d-4aa2-a49a-fd021c2d4df2\") " pod="openshift-console-operator/console-operator-7866497494-r8cn4" Jul 16 16:51:34.457119 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.457103 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bd2d4507-465f-4002-8953-9e289841cdbb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-5tsmr\" (UID: \"bd2d4507-465f-4002-8953-9e289841cdbb\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5tsmr" Jul 16 16:51:34.457157 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.457128 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f637d961-672d-4aa2-a49a-fd021c2d4df2-serving-cert\") pod \"console-operator-7866497494-r8cn4\" (UID: \"f637d961-672d-4aa2-a49a-fd021c2d4df2\") " pod="openshift-console-operator/console-operator-7866497494-r8cn4" Jul 16 16:51:34.457157 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.457150 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jp4q\" (UniqueName: \"kubernetes.io/projected/d2bdea19-f5df-4297-8304-da19a8cc7a62-kube-api-access-6jp4q\") pod \"router-default-57bc6f48c6-kt2wd\" (UID: \"d2bdea19-f5df-4297-8304-da19a8cc7a62\") " pod="openshift-ingress/router-default-57bc6f48c6-kt2wd" Jul 16 16:51:34.457217 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.457166 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2bdea19-f5df-4297-8304-da19a8cc7a62-service-ca-bundle\") pod \"router-default-57bc6f48c6-kt2wd\" (UID: \"d2bdea19-f5df-4297-8304-da19a8cc7a62\") " pod="openshift-ingress/router-default-57bc6f48c6-kt2wd" Jul 16 16:51:34.457217 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.457184 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37bad57e-950d-4a6a-984b-19f240be539b-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-5t9rp\" (UID: \"37bad57e-950d-4a6a-984b-19f240be539b\") " pod="openshift-insights/insights-operator-68d9dd94cf-5t9rp" Jul 16 16:51:34.457217 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.457204 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/32cd8d7c-6e88-483e-a5d7-f028311b4be3-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-fjsn9\" (UID: \"32cd8d7c-6e88-483e-a5d7-f028311b4be3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fjsn9" Jul 16 16:51:34.457324 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.457248 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/37bad57e-950d-4a6a-984b-19f240be539b-snapshots\") pod \"insights-operator-68d9dd94cf-5t9rp\" (UID: \"37bad57e-950d-4a6a-984b-19f240be539b\") " pod="openshift-insights/insights-operator-68d9dd94cf-5t9rp" Jul 16 16:51:34.457324 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:34.457292 2571 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 16:51:34.457384 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:34.457350 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd2d4507-465f-4002-8953-9e289841cdbb-cluster-monitoring-operator-tls podName:bd2d4507-465f-4002-8953-9e289841cdbb nodeName:}" failed. No retries permitted until 2026-07-16 16:51:34.957332816 +0000 UTC m=+46.698908094 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/bd2d4507-465f-4002-8953-9e289841cdbb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-5tsmr" (UID: "bd2d4507-465f-4002-8953-9e289841cdbb") : secret "cluster-monitoring-operator-tls" not found Jul 16 16:51:34.457384 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.457366 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrqx9\" (UniqueName: \"kubernetes.io/projected/bd2d4507-465f-4002-8953-9e289841cdbb-kube-api-access-zrqx9\") pod \"cluster-monitoring-operator-5f7f494dd4-5tsmr\" (UID: \"bd2d4507-465f-4002-8953-9e289841cdbb\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5tsmr" Jul 16 16:51:34.457452 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.457388 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/bd2d4507-465f-4002-8953-9e289841cdbb-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-5tsmr\" (UID: \"bd2d4507-465f-4002-8953-9e289841cdbb\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5tsmr" Jul 16 16:51:34.457452 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.457412 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqfc9\" (UniqueName: \"kubernetes.io/projected/37bad57e-950d-4a6a-984b-19f240be539b-kube-api-access-qqfc9\") pod \"insights-operator-68d9dd94cf-5t9rp\" (UID: \"37bad57e-950d-4a6a-984b-19f240be539b\") " pod="openshift-insights/insights-operator-68d9dd94cf-5t9rp" Jul 16 16:51:34.457531 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.457448 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhn6d\" (UniqueName: \"kubernetes.io/projected/32cd8d7c-6e88-483e-a5d7-f028311b4be3-kube-api-access-nhn6d\") pod \"kube-storage-version-migrator-operator-799cb7f958-fjsn9\" (UID: \"32cd8d7c-6e88-483e-a5d7-f028311b4be3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fjsn9" Jul 16 16:51:34.457531 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.457473 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v24sv\" (UniqueName: \"kubernetes.io/projected/f637d961-672d-4aa2-a49a-fd021c2d4df2-kube-api-access-v24sv\") pod \"console-operator-7866497494-r8cn4\" (UID: \"f637d961-672d-4aa2-a49a-fd021c2d4df2\") " pod="openshift-console-operator/console-operator-7866497494-r8cn4" Jul 16 16:51:34.457531 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.457504 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/37bad57e-950d-4a6a-984b-19f240be539b-serving-cert\") pod \"insights-operator-68d9dd94cf-5t9rp\" (UID: \"37bad57e-950d-4a6a-984b-19f240be539b\") " pod="openshift-insights/insights-operator-68d9dd94cf-5t9rp" Jul 16 16:51:34.457689 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.457550 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2bdea19-f5df-4297-8304-da19a8cc7a62-metrics-certs\") pod \"router-default-57bc6f48c6-kt2wd\" (UID: \"d2bdea19-f5df-4297-8304-da19a8cc7a62\") " pod="openshift-ingress/router-default-57bc6f48c6-kt2wd" Jul 16 16:51:34.457689 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.457574 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f637d961-672d-4aa2-a49a-fd021c2d4df2-trusted-ca\") pod \"console-operator-7866497494-r8cn4\" (UID: \"f637d961-672d-4aa2-a49a-fd021c2d4df2\") " pod="openshift-console-operator/console-operator-7866497494-r8cn4" Jul 16 16:51:34.457689 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.457592 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37bad57e-950d-4a6a-984b-19f240be539b-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-5t9rp\" (UID: \"37bad57e-950d-4a6a-984b-19f240be539b\") " pod="openshift-insights/insights-operator-68d9dd94cf-5t9rp" Jul 16 16:51:34.457689 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.457616 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/32cd8d7c-6e88-483e-a5d7-f028311b4be3-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-fjsn9\" (UID: \"32cd8d7c-6e88-483e-a5d7-f028311b4be3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fjsn9" Jul 16 16:51:34.457689 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.457668 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghnrp\" (UniqueName: \"kubernetes.io/projected/d67e5a27-8c54-42e3-af31-f7abfd8ddf95-kube-api-access-ghnrp\") pod \"cluster-samples-operator-65db858894-5ctvn\" (UID: \"d67e5a27-8c54-42e3-af31-f7abfd8ddf95\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5ctvn" Jul 16 16:51:34.457906 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.457718 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d2bdea19-f5df-4297-8304-da19a8cc7a62-default-certificate\") pod \"router-default-57bc6f48c6-kt2wd\" (UID: \"d2bdea19-f5df-4297-8304-da19a8cc7a62\") " pod="openshift-ingress/router-default-57bc6f48c6-kt2wd" Jul 16 16:51:34.457906 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.457765 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37bad57e-950d-4a6a-984b-19f240be539b-tmp\") pod \"insights-operator-68d9dd94cf-5t9rp\" (UID: \"37bad57e-950d-4a6a-984b-19f240be539b\") " pod="openshift-insights/insights-operator-68d9dd94cf-5t9rp" Jul 16 16:51:34.458060 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.458044 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/bd2d4507-465f-4002-8953-9e289841cdbb-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-5tsmr\" (UID: \"bd2d4507-465f-4002-8953-9e289841cdbb\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5tsmr" Jul 16 16:51:34.458093 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.458071 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/32cd8d7c-6e88-483e-a5d7-f028311b4be3-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-fjsn9\" (UID: \"32cd8d7c-6e88-483e-a5d7-f028311b4be3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fjsn9" Jul 16 16:51:34.459195 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.459172 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/32cd8d7c-6e88-483e-a5d7-f028311b4be3-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-fjsn9\" (UID: \"32cd8d7c-6e88-483e-a5d7-f028311b4be3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fjsn9" Jul 16 16:51:34.465494 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.465477 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-6zvr8"] Jul 16 16:51:34.465568 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.465563 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-6zvr8" Jul 16 16:51:34.467160 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.467132 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrqx9\" (UniqueName: \"kubernetes.io/projected/bd2d4507-465f-4002-8953-9e289841cdbb-kube-api-access-zrqx9\") pod \"cluster-monitoring-operator-5f7f494dd4-5tsmr\" (UID: \"bd2d4507-465f-4002-8953-9e289841cdbb\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5tsmr" Jul 16 16:51:34.467842 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.467822 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhn6d\" (UniqueName: \"kubernetes.io/projected/32cd8d7c-6e88-483e-a5d7-f028311b4be3-kube-api-access-nhn6d\") pod \"kube-storage-version-migrator-operator-799cb7f958-fjsn9\" (UID: \"32cd8d7c-6e88-483e-a5d7-f028311b4be3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fjsn9" Jul 16 16:51:34.468092 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.468077 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-47jfg\"" Jul 16 16:51:34.468191 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.468171 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 16 16:51:34.468307 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.468292 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 16 16:51:34.468343 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.468310 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 16 16:51:34.468343 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.468329 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 16 16:51:34.486637 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.486621 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-ct89n" Jul 16 16:51:34.559068 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.558044 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/37bad57e-950d-4a6a-984b-19f240be539b-snapshots\") pod \"insights-operator-68d9dd94cf-5t9rp\" (UID: \"37bad57e-950d-4a6a-984b-19f240be539b\") " pod="openshift-insights/insights-operator-68d9dd94cf-5t9rp" Jul 16 16:51:34.559068 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.558080 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gc4lb\" (UniqueName: \"kubernetes.io/projected/3d9162b4-f5a7-4aff-afd6-73383ad17f4a-kube-api-access-gc4lb\") pod \"service-ca-operator-7695fc6c8c-6zvr8\" (UID: \"3d9162b4-f5a7-4aff-afd6-73383ad17f4a\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-6zvr8" Jul 16 16:51:34.559068 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.558101 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqfc9\" (UniqueName: \"kubernetes.io/projected/37bad57e-950d-4a6a-984b-19f240be539b-kube-api-access-qqfc9\") pod \"insights-operator-68d9dd94cf-5t9rp\" (UID: \"37bad57e-950d-4a6a-984b-19f240be539b\") " pod="openshift-insights/insights-operator-68d9dd94cf-5t9rp" Jul 16 16:51:34.559068 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.558131 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v24sv\" (UniqueName: \"kubernetes.io/projected/f637d961-672d-4aa2-a49a-fd021c2d4df2-kube-api-access-v24sv\") pod \"console-operator-7866497494-r8cn4\" (UID: \"f637d961-672d-4aa2-a49a-fd021c2d4df2\") " pod="openshift-console-operator/console-operator-7866497494-r8cn4" Jul 16 16:51:34.559068 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.558153 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/37bad57e-950d-4a6a-984b-19f240be539b-serving-cert\") pod \"insights-operator-68d9dd94cf-5t9rp\" (UID: \"37bad57e-950d-4a6a-984b-19f240be539b\") " pod="openshift-insights/insights-operator-68d9dd94cf-5t9rp" Jul 16 16:51:34.559068 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.558169 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2bdea19-f5df-4297-8304-da19a8cc7a62-metrics-certs\") pod \"router-default-57bc6f48c6-kt2wd\" (UID: \"d2bdea19-f5df-4297-8304-da19a8cc7a62\") " pod="openshift-ingress/router-default-57bc6f48c6-kt2wd" Jul 16 16:51:34.559068 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.558191 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f637d961-672d-4aa2-a49a-fd021c2d4df2-trusted-ca\") pod \"console-operator-7866497494-r8cn4\" (UID: \"f637d961-672d-4aa2-a49a-fd021c2d4df2\") " pod="openshift-console-operator/console-operator-7866497494-r8cn4" Jul 16 16:51:34.559068 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.558208 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37bad57e-950d-4a6a-984b-19f240be539b-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-5t9rp\" (UID: \"37bad57e-950d-4a6a-984b-19f240be539b\") " pod="openshift-insights/insights-operator-68d9dd94cf-5t9rp" Jul 16 16:51:34.559068 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.558228 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghnrp\" (UniqueName: \"kubernetes.io/projected/d67e5a27-8c54-42e3-af31-f7abfd8ddf95-kube-api-access-ghnrp\") pod \"cluster-samples-operator-65db858894-5ctvn\" (UID: \"d67e5a27-8c54-42e3-af31-f7abfd8ddf95\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5ctvn" Jul 16 16:51:34.559068 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.558250 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d2bdea19-f5df-4297-8304-da19a8cc7a62-default-certificate\") pod \"router-default-57bc6f48c6-kt2wd\" (UID: \"d2bdea19-f5df-4297-8304-da19a8cc7a62\") " pod="openshift-ingress/router-default-57bc6f48c6-kt2wd" Jul 16 16:51:34.559068 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.558265 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37bad57e-950d-4a6a-984b-19f240be539b-tmp\") pod \"insights-operator-68d9dd94cf-5t9rp\" (UID: \"37bad57e-950d-4a6a-984b-19f240be539b\") " pod="openshift-insights/insights-operator-68d9dd94cf-5t9rp" Jul 16 16:51:34.559068 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.558281 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d2bdea19-f5df-4297-8304-da19a8cc7a62-stats-auth\") pod \"router-default-57bc6f48c6-kt2wd\" (UID: \"d2bdea19-f5df-4297-8304-da19a8cc7a62\") " pod="openshift-ingress/router-default-57bc6f48c6-kt2wd" Jul 16 16:51:34.559068 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.558297 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d67e5a27-8c54-42e3-af31-f7abfd8ddf95-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-5ctvn\" (UID: \"d67e5a27-8c54-42e3-af31-f7abfd8ddf95\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5ctvn" Jul 16 16:51:34.559068 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.558313 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f637d961-672d-4aa2-a49a-fd021c2d4df2-config\") pod \"console-operator-7866497494-r8cn4\" (UID: \"f637d961-672d-4aa2-a49a-fd021c2d4df2\") " pod="openshift-console-operator/console-operator-7866497494-r8cn4" Jul 16 16:51:34.559068 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.558343 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3d9162b4-f5a7-4aff-afd6-73383ad17f4a-serving-cert\") pod \"service-ca-operator-7695fc6c8c-6zvr8\" (UID: \"3d9162b4-f5a7-4aff-afd6-73383ad17f4a\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-6zvr8" Jul 16 16:51:34.559068 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.558366 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f637d961-672d-4aa2-a49a-fd021c2d4df2-serving-cert\") pod \"console-operator-7866497494-r8cn4\" (UID: \"f637d961-672d-4aa2-a49a-fd021c2d4df2\") " pod="openshift-console-operator/console-operator-7866497494-r8cn4" Jul 16 16:51:34.559940 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.558381 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jp4q\" (UniqueName: \"kubernetes.io/projected/d2bdea19-f5df-4297-8304-da19a8cc7a62-kube-api-access-6jp4q\") pod \"router-default-57bc6f48c6-kt2wd\" (UID: \"d2bdea19-f5df-4297-8304-da19a8cc7a62\") " pod="openshift-ingress/router-default-57bc6f48c6-kt2wd" Jul 16 16:51:34.559940 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.558397 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2bdea19-f5df-4297-8304-da19a8cc7a62-service-ca-bundle\") pod \"router-default-57bc6f48c6-kt2wd\" (UID: \"d2bdea19-f5df-4297-8304-da19a8cc7a62\") " pod="openshift-ingress/router-default-57bc6f48c6-kt2wd" Jul 16 16:51:34.559940 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.558416 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37bad57e-950d-4a6a-984b-19f240be539b-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-5t9rp\" (UID: \"37bad57e-950d-4a6a-984b-19f240be539b\") " pod="openshift-insights/insights-operator-68d9dd94cf-5t9rp" Jul 16 16:51:34.559940 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.558431 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3d9162b4-f5a7-4aff-afd6-73383ad17f4a-config\") pod \"service-ca-operator-7695fc6c8c-6zvr8\" (UID: \"3d9162b4-f5a7-4aff-afd6-73383ad17f4a\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-6zvr8" Jul 16 16:51:34.559940 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.558994 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/37bad57e-950d-4a6a-984b-19f240be539b-snapshots\") pod \"insights-operator-68d9dd94cf-5t9rp\" (UID: \"37bad57e-950d-4a6a-984b-19f240be539b\") " pod="openshift-insights/insights-operator-68d9dd94cf-5t9rp" Jul 16 16:51:34.559940 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:34.559122 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d2bdea19-f5df-4297-8304-da19a8cc7a62-service-ca-bundle podName:d2bdea19-f5df-4297-8304-da19a8cc7a62 nodeName:}" failed. No retries permitted until 2026-07-16 16:51:35.059106646 +0000 UTC m=+46.800681922 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d2bdea19-f5df-4297-8304-da19a8cc7a62-service-ca-bundle") pod "router-default-57bc6f48c6-kt2wd" (UID: "d2bdea19-f5df-4297-8304-da19a8cc7a62") : configmap references non-existent config key: service-ca.crt Jul 16 16:51:34.560276 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.560097 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37bad57e-950d-4a6a-984b-19f240be539b-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-5t9rp\" (UID: \"37bad57e-950d-4a6a-984b-19f240be539b\") " pod="openshift-insights/insights-operator-68d9dd94cf-5t9rp" Jul 16 16:51:34.560276 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.560171 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f637d961-672d-4aa2-a49a-fd021c2d4df2-trusted-ca\") pod \"console-operator-7866497494-r8cn4\" (UID: \"f637d961-672d-4aa2-a49a-fd021c2d4df2\") " pod="openshift-console-operator/console-operator-7866497494-r8cn4" Jul 16 16:51:34.560276 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:34.560264 2571 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 16:51:34.560483 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:34.560312 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2bdea19-f5df-4297-8304-da19a8cc7a62-metrics-certs podName:d2bdea19-f5df-4297-8304-da19a8cc7a62 nodeName:}" failed. No retries permitted until 2026-07-16 16:51:35.060295879 +0000 UTC m=+46.801871176 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2bdea19-f5df-4297-8304-da19a8cc7a62-metrics-certs") pod "router-default-57bc6f48c6-kt2wd" (UID: "d2bdea19-f5df-4297-8304-da19a8cc7a62") : secret "router-metrics-certs-default" not found Jul 16 16:51:34.560598 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.560569 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37bad57e-950d-4a6a-984b-19f240be539b-tmp\") pod \"insights-operator-68d9dd94cf-5t9rp\" (UID: \"37bad57e-950d-4a6a-984b-19f240be539b\") " pod="openshift-insights/insights-operator-68d9dd94cf-5t9rp" Jul 16 16:51:34.561083 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.561057 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f637d961-672d-4aa2-a49a-fd021c2d4df2-config\") pod \"console-operator-7866497494-r8cn4\" (UID: \"f637d961-672d-4aa2-a49a-fd021c2d4df2\") " pod="openshift-console-operator/console-operator-7866497494-r8cn4" Jul 16 16:51:34.561187 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.561105 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37bad57e-950d-4a6a-984b-19f240be539b-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-5t9rp\" (UID: \"37bad57e-950d-4a6a-984b-19f240be539b\") " pod="openshift-insights/insights-operator-68d9dd94cf-5t9rp" Jul 16 16:51:34.561312 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:34.561296 2571 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 16:51:34.561379 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:34.561367 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d67e5a27-8c54-42e3-af31-f7abfd8ddf95-samples-operator-tls podName:d67e5a27-8c54-42e3-af31-f7abfd8ddf95 nodeName:}" failed. No retries permitted until 2026-07-16 16:51:35.061351648 +0000 UTC m=+46.802926925 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d67e5a27-8c54-42e3-af31-f7abfd8ddf95-samples-operator-tls") pod "cluster-samples-operator-65db858894-5ctvn" (UID: "d67e5a27-8c54-42e3-af31-f7abfd8ddf95") : secret "samples-operator-tls" not found Jul 16 16:51:34.562926 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.562901 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d2bdea19-f5df-4297-8304-da19a8cc7a62-default-certificate\") pod \"router-default-57bc6f48c6-kt2wd\" (UID: \"d2bdea19-f5df-4297-8304-da19a8cc7a62\") " pod="openshift-ingress/router-default-57bc6f48c6-kt2wd" Jul 16 16:51:34.563346 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.563297 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f637d961-672d-4aa2-a49a-fd021c2d4df2-serving-cert\") pod \"console-operator-7866497494-r8cn4\" (UID: \"f637d961-672d-4aa2-a49a-fd021c2d4df2\") " pod="openshift-console-operator/console-operator-7866497494-r8cn4" Jul 16 16:51:34.563440 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.563398 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d2bdea19-f5df-4297-8304-da19a8cc7a62-stats-auth\") pod \"router-default-57bc6f48c6-kt2wd\" (UID: \"d2bdea19-f5df-4297-8304-da19a8cc7a62\") " pod="openshift-ingress/router-default-57bc6f48c6-kt2wd" Jul 16 16:51:34.564822 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.564799 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/37bad57e-950d-4a6a-984b-19f240be539b-serving-cert\") pod \"insights-operator-68d9dd94cf-5t9rp\" (UID: \"37bad57e-950d-4a6a-984b-19f240be539b\") " pod="openshift-insights/insights-operator-68d9dd94cf-5t9rp" Jul 16 16:51:34.568800 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.568769 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqfc9\" (UniqueName: \"kubernetes.io/projected/37bad57e-950d-4a6a-984b-19f240be539b-kube-api-access-qqfc9\") pod \"insights-operator-68d9dd94cf-5t9rp\" (UID: \"37bad57e-950d-4a6a-984b-19f240be539b\") " pod="openshift-insights/insights-operator-68d9dd94cf-5t9rp" Jul 16 16:51:34.570104 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.570083 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v24sv\" (UniqueName: \"kubernetes.io/projected/f637d961-672d-4aa2-a49a-fd021c2d4df2-kube-api-access-v24sv\") pod \"console-operator-7866497494-r8cn4\" (UID: \"f637d961-672d-4aa2-a49a-fd021c2d4df2\") " pod="openshift-console-operator/console-operator-7866497494-r8cn4" Jul 16 16:51:34.570238 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.570220 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jp4q\" (UniqueName: \"kubernetes.io/projected/d2bdea19-f5df-4297-8304-da19a8cc7a62-kube-api-access-6jp4q\") pod \"router-default-57bc6f48c6-kt2wd\" (UID: \"d2bdea19-f5df-4297-8304-da19a8cc7a62\") " pod="openshift-ingress/router-default-57bc6f48c6-kt2wd" Jul 16 16:51:34.570355 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.570337 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghnrp\" (UniqueName: \"kubernetes.io/projected/d67e5a27-8c54-42e3-af31-f7abfd8ddf95-kube-api-access-ghnrp\") pod \"cluster-samples-operator-65db858894-5ctvn\" (UID: \"d67e5a27-8c54-42e3-af31-f7abfd8ddf95\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5ctvn" Jul 16 16:51:34.597115 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.597096 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fjsn9" Jul 16 16:51:34.624702 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.624655 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-ct89n"] Jul 16 16:51:34.627039 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:51:34.627022 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1896b4d7_17ae_46be_98bc_54a304f42eda.slice/crio-39b802a66b0db2f8ccf19e0e19a394d937c7da3c2bbae5bb179f6fb0d4dc58cc WatchSource:0}: Error finding container 39b802a66b0db2f8ccf19e0e19a394d937c7da3c2bbae5bb179f6fb0d4dc58cc: Status 404 returned error can't find the container with id 39b802a66b0db2f8ccf19e0e19a394d937c7da3c2bbae5bb179f6fb0d4dc58cc Jul 16 16:51:34.659609 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.659555 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gc4lb\" (UniqueName: \"kubernetes.io/projected/3d9162b4-f5a7-4aff-afd6-73383ad17f4a-kube-api-access-gc4lb\") pod \"service-ca-operator-7695fc6c8c-6zvr8\" (UID: \"3d9162b4-f5a7-4aff-afd6-73383ad17f4a\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-6zvr8" Jul 16 16:51:34.659702 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.659658 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3d9162b4-f5a7-4aff-afd6-73383ad17f4a-serving-cert\") pod \"service-ca-operator-7695fc6c8c-6zvr8\" (UID: \"3d9162b4-f5a7-4aff-afd6-73383ad17f4a\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-6zvr8" Jul 16 16:51:34.659702 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.659695 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3d9162b4-f5a7-4aff-afd6-73383ad17f4a-config\") pod \"service-ca-operator-7695fc6c8c-6zvr8\" (UID: \"3d9162b4-f5a7-4aff-afd6-73383ad17f4a\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-6zvr8" Jul 16 16:51:34.660146 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.660128 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3d9162b4-f5a7-4aff-afd6-73383ad17f4a-config\") pod \"service-ca-operator-7695fc6c8c-6zvr8\" (UID: \"3d9162b4-f5a7-4aff-afd6-73383ad17f4a\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-6zvr8" Jul 16 16:51:34.661878 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.661859 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3d9162b4-f5a7-4aff-afd6-73383ad17f4a-serving-cert\") pod \"service-ca-operator-7695fc6c8c-6zvr8\" (UID: \"3d9162b4-f5a7-4aff-afd6-73383ad17f4a\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-6zvr8" Jul 16 16:51:34.668135 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.668110 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gc4lb\" (UniqueName: \"kubernetes.io/projected/3d9162b4-f5a7-4aff-afd6-73383ad17f4a-kube-api-access-gc4lb\") pod \"service-ca-operator-7695fc6c8c-6zvr8\" (UID: \"3d9162b4-f5a7-4aff-afd6-73383ad17f4a\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-6zvr8" Jul 16 16:51:34.685607 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.685587 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-5t9rp" Jul 16 16:51:34.711511 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.711491 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-r8cn4" Jul 16 16:51:34.743168 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.743147 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fjsn9"] Jul 16 16:51:34.745277 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:51:34.745254 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32cd8d7c_6e88_483e_a5d7_f028311b4be3.slice/crio-a6590f69b8f0a930e501517992218cc200d053b476035e72b499905ff29db841 WatchSource:0}: Error finding container a6590f69b8f0a930e501517992218cc200d053b476035e72b499905ff29db841: Status 404 returned error can't find the container with id a6590f69b8f0a930e501517992218cc200d053b476035e72b499905ff29db841 Jul 16 16:51:34.779858 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.779838 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-6zvr8" Jul 16 16:51:34.830079 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.830050 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-5t9rp"] Jul 16 16:51:34.831739 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:51:34.831686 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37bad57e_950d_4a6a_984b_19f240be539b.slice/crio-e4b286762cf8de7265b47dbd2c1a61a25655c411e2dab129211c4d4b8e98580a WatchSource:0}: Error finding container e4b286762cf8de7265b47dbd2c1a61a25655c411e2dab129211c4d4b8e98580a: Status 404 returned error can't find the container with id e4b286762cf8de7265b47dbd2c1a61a25655c411e2dab129211c4d4b8e98580a Jul 16 16:51:34.873634 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.873612 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-r8cn4"] Jul 16 16:51:34.876226 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:51:34.876203 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf637d961_672d_4aa2_a49a_fd021c2d4df2.slice/crio-0482ac1faf32fe7a7f1e435e9f28cd3d165c3f0448dd675f5af12f9f3d7398c2 WatchSource:0}: Error finding container 0482ac1faf32fe7a7f1e435e9f28cd3d165c3f0448dd675f5af12f9f3d7398c2: Status 404 returned error can't find the container with id 0482ac1faf32fe7a7f1e435e9f28cd3d165c3f0448dd675f5af12f9f3d7398c2 Jul 16 16:51:34.927800 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.927776 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-6zvr8"] Jul 16 16:51:34.932286 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:51:34.932253 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d9162b4_f5a7_4aff_afd6_73383ad17f4a.slice/crio-b9f87a683365889e87c7c4cbac8ddb0b5e9830ca8f2a95e5fc694feaf6ed0f0f WatchSource:0}: Error finding container b9f87a683365889e87c7c4cbac8ddb0b5e9830ca8f2a95e5fc694feaf6ed0f0f: Status 404 returned error can't find the container with id b9f87a683365889e87c7c4cbac8ddb0b5e9830ca8f2a95e5fc694feaf6ed0f0f Jul 16 16:51:34.963057 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:34.963033 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bd2d4507-465f-4002-8953-9e289841cdbb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-5tsmr\" (UID: \"bd2d4507-465f-4002-8953-9e289841cdbb\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5tsmr" Jul 16 16:51:34.963146 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:34.963132 2571 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 16:51:34.963184 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:34.963179 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd2d4507-465f-4002-8953-9e289841cdbb-cluster-monitoring-operator-tls podName:bd2d4507-465f-4002-8953-9e289841cdbb nodeName:}" failed. No retries permitted until 2026-07-16 16:51:35.963166397 +0000 UTC m=+47.704741672 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/bd2d4507-465f-4002-8953-9e289841cdbb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-5tsmr" (UID: "bd2d4507-465f-4002-8953-9e289841cdbb") : secret "cluster-monitoring-operator-tls" not found Jul 16 16:51:35.023092 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:35.023067 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-r8cn4" event={"ID":"f637d961-672d-4aa2-a49a-fd021c2d4df2","Type":"ContainerStarted","Data":"0482ac1faf32fe7a7f1e435e9f28cd3d165c3f0448dd675f5af12f9f3d7398c2"} Jul 16 16:51:35.024179 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:35.024155 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fjsn9" event={"ID":"32cd8d7c-6e88-483e-a5d7-f028311b4be3","Type":"ContainerStarted","Data":"a6590f69b8f0a930e501517992218cc200d053b476035e72b499905ff29db841"} Jul 16 16:51:35.025201 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:35.025174 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-5t9rp" event={"ID":"37bad57e-950d-4a6a-984b-19f240be539b","Type":"ContainerStarted","Data":"e4b286762cf8de7265b47dbd2c1a61a25655c411e2dab129211c4d4b8e98580a"} Jul 16 16:51:35.026173 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:35.026153 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-ct89n" event={"ID":"1896b4d7-17ae-46be-98bc-54a304f42eda","Type":"ContainerStarted","Data":"39b802a66b0db2f8ccf19e0e19a394d937c7da3c2bbae5bb179f6fb0d4dc58cc"} Jul 16 16:51:35.027094 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:35.027075 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-6zvr8" event={"ID":"3d9162b4-f5a7-4aff-afd6-73383ad17f4a","Type":"ContainerStarted","Data":"b9f87a683365889e87c7c4cbac8ddb0b5e9830ca8f2a95e5fc694feaf6ed0f0f"} Jul 16 16:51:35.063982 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:35.063960 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2bdea19-f5df-4297-8304-da19a8cc7a62-metrics-certs\") pod \"router-default-57bc6f48c6-kt2wd\" (UID: \"d2bdea19-f5df-4297-8304-da19a8cc7a62\") " pod="openshift-ingress/router-default-57bc6f48c6-kt2wd" Jul 16 16:51:35.064054 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:35.064009 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d67e5a27-8c54-42e3-af31-f7abfd8ddf95-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-5ctvn\" (UID: \"d67e5a27-8c54-42e3-af31-f7abfd8ddf95\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5ctvn" Jul 16 16:51:35.064054 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:35.064050 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2bdea19-f5df-4297-8304-da19a8cc7a62-service-ca-bundle\") pod \"router-default-57bc6f48c6-kt2wd\" (UID: \"d2bdea19-f5df-4297-8304-da19a8cc7a62\") " pod="openshift-ingress/router-default-57bc6f48c6-kt2wd" Jul 16 16:51:35.064153 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:35.064084 2571 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 16:51:35.064203 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:35.064180 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2bdea19-f5df-4297-8304-da19a8cc7a62-metrics-certs podName:d2bdea19-f5df-4297-8304-da19a8cc7a62 nodeName:}" failed. No retries permitted until 2026-07-16 16:51:36.06411701 +0000 UTC m=+47.805692285 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2bdea19-f5df-4297-8304-da19a8cc7a62-metrics-certs") pod "router-default-57bc6f48c6-kt2wd" (UID: "d2bdea19-f5df-4297-8304-da19a8cc7a62") : secret "router-metrics-certs-default" not found Jul 16 16:51:35.064203 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:35.064198 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d2bdea19-f5df-4297-8304-da19a8cc7a62-service-ca-bundle podName:d2bdea19-f5df-4297-8304-da19a8cc7a62 nodeName:}" failed. No retries permitted until 2026-07-16 16:51:36.064190285 +0000 UTC m=+47.805765559 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d2bdea19-f5df-4297-8304-da19a8cc7a62-service-ca-bundle") pod "router-default-57bc6f48c6-kt2wd" (UID: "d2bdea19-f5df-4297-8304-da19a8cc7a62") : configmap references non-existent config key: service-ca.crt Jul 16 16:51:35.064307 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:35.064227 2571 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 16:51:35.064307 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:35.064283 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d67e5a27-8c54-42e3-af31-f7abfd8ddf95-samples-operator-tls podName:d67e5a27-8c54-42e3-af31-f7abfd8ddf95 nodeName:}" failed. No retries permitted until 2026-07-16 16:51:36.064265239 +0000 UTC m=+47.805840528 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d67e5a27-8c54-42e3-af31-f7abfd8ddf95-samples-operator-tls") pod "cluster-samples-operator-65db858894-5ctvn" (UID: "d67e5a27-8c54-42e3-af31-f7abfd8ddf95") : secret "samples-operator-tls" not found Jul 16 16:51:35.366932 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:35.366307 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-lbhfp"] Jul 16 16:51:35.384612 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:35.384580 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-lbhfp"] Jul 16 16:51:35.384716 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:35.384678 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-lbhfp" Jul 16 16:51:35.388055 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:35.387872 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-r4nsj\"" Jul 16 16:51:35.467318 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:35.467231 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p49wc\" (UniqueName: \"kubernetes.io/projected/ae213c51-b545-4cf5-a244-0f10ff891fab-kube-api-access-p49wc\") pod \"network-check-source-869dcfcdcd-lbhfp\" (UID: \"ae213c51-b545-4cf5-a244-0f10ff891fab\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-lbhfp" Jul 16 16:51:35.568685 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:35.568654 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p49wc\" (UniqueName: \"kubernetes.io/projected/ae213c51-b545-4cf5-a244-0f10ff891fab-kube-api-access-p49wc\") pod \"network-check-source-869dcfcdcd-lbhfp\" (UID: \"ae213c51-b545-4cf5-a244-0f10ff891fab\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-lbhfp" Jul 16 16:51:35.582276 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:35.582212 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p49wc\" (UniqueName: \"kubernetes.io/projected/ae213c51-b545-4cf5-a244-0f10ff891fab-kube-api-access-p49wc\") pod \"network-check-source-869dcfcdcd-lbhfp\" (UID: \"ae213c51-b545-4cf5-a244-0f10ff891fab\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-lbhfp" Jul 16 16:51:35.720553 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:35.720522 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-lbhfp" Jul 16 16:51:35.972664 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:35.972591 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bd2d4507-465f-4002-8953-9e289841cdbb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-5tsmr\" (UID: \"bd2d4507-465f-4002-8953-9e289841cdbb\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5tsmr" Jul 16 16:51:35.972822 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:35.972807 2571 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 16:51:35.972886 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:35.972874 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd2d4507-465f-4002-8953-9e289841cdbb-cluster-monitoring-operator-tls podName:bd2d4507-465f-4002-8953-9e289841cdbb nodeName:}" failed. No retries permitted until 2026-07-16 16:51:37.972853034 +0000 UTC m=+49.714428313 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/bd2d4507-465f-4002-8953-9e289841cdbb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-5tsmr" (UID: "bd2d4507-465f-4002-8953-9e289841cdbb") : secret "cluster-monitoring-operator-tls" not found Jul 16 16:51:36.074194 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:36.074126 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d67e5a27-8c54-42e3-af31-f7abfd8ddf95-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-5ctvn\" (UID: \"d67e5a27-8c54-42e3-af31-f7abfd8ddf95\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5ctvn" Jul 16 16:51:36.074345 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:36.074209 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2bdea19-f5df-4297-8304-da19a8cc7a62-service-ca-bundle\") pod \"router-default-57bc6f48c6-kt2wd\" (UID: \"d2bdea19-f5df-4297-8304-da19a8cc7a62\") " pod="openshift-ingress/router-default-57bc6f48c6-kt2wd" Jul 16 16:51:36.074345 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:36.074300 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2bdea19-f5df-4297-8304-da19a8cc7a62-metrics-certs\") pod \"router-default-57bc6f48c6-kt2wd\" (UID: \"d2bdea19-f5df-4297-8304-da19a8cc7a62\") " pod="openshift-ingress/router-default-57bc6f48c6-kt2wd" Jul 16 16:51:36.074345 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:36.074322 2571 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 16:51:36.074517 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:36.074391 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d67e5a27-8c54-42e3-af31-f7abfd8ddf95-samples-operator-tls podName:d67e5a27-8c54-42e3-af31-f7abfd8ddf95 nodeName:}" failed. No retries permitted until 2026-07-16 16:51:38.074369929 +0000 UTC m=+49.815945230 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d67e5a27-8c54-42e3-af31-f7abfd8ddf95-samples-operator-tls") pod "cluster-samples-operator-65db858894-5ctvn" (UID: "d67e5a27-8c54-42e3-af31-f7abfd8ddf95") : secret "samples-operator-tls" not found Jul 16 16:51:36.074517 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:36.074406 2571 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 16:51:36.074517 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:36.074452 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2bdea19-f5df-4297-8304-da19a8cc7a62-metrics-certs podName:d2bdea19-f5df-4297-8304-da19a8cc7a62 nodeName:}" failed. No retries permitted until 2026-07-16 16:51:38.074437213 +0000 UTC m=+49.816012495 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2bdea19-f5df-4297-8304-da19a8cc7a62-metrics-certs") pod "router-default-57bc6f48c6-kt2wd" (UID: "d2bdea19-f5df-4297-8304-da19a8cc7a62") : secret "router-metrics-certs-default" not found Jul 16 16:51:36.074517 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:36.074511 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d2bdea19-f5df-4297-8304-da19a8cc7a62-service-ca-bundle podName:d2bdea19-f5df-4297-8304-da19a8cc7a62 nodeName:}" failed. No retries permitted until 2026-07-16 16:51:38.074501064 +0000 UTC m=+49.816076351 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d2bdea19-f5df-4297-8304-da19a8cc7a62-service-ca-bundle") pod "router-default-57bc6f48c6-kt2wd" (UID: "d2bdea19-f5df-4297-8304-da19a8cc7a62") : configmap references non-existent config key: service-ca.crt Jul 16 16:51:36.279070 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:36.278931 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-lbhfp"] Jul 16 16:51:36.283840 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:51:36.283805 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae213c51_b545_4cf5_a244_0f10ff891fab.slice/crio-14bd64bd0136609a4f37d41e594d6851e2c89b9a641211d26cf575fc5f6d3c98 WatchSource:0}: Error finding container 14bd64bd0136609a4f37d41e594d6851e2c89b9a641211d26cf575fc5f6d3c98: Status 404 returned error can't find the container with id 14bd64bd0136609a4f37d41e594d6851e2c89b9a641211d26cf575fc5f6d3c98 Jul 16 16:51:37.035373 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:37.035338 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-ct89n" event={"ID":"1896b4d7-17ae-46be-98bc-54a304f42eda","Type":"ContainerStarted","Data":"59d58d615106005cd93ec65eaefde38bab532978f5d6cc755b21c938b2ffb97c"} Jul 16 16:51:37.036703 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:37.036668 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-lbhfp" event={"ID":"ae213c51-b545-4cf5-a244-0f10ff891fab","Type":"ContainerStarted","Data":"14bd64bd0136609a4f37d41e594d6851e2c89b9a641211d26cf575fc5f6d3c98"} Jul 16 16:51:37.054747 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:37.054681 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-ct89n" podStartSLOduration=1.186831013 podStartE2EDuration="3.05466088s" totalImagesPullingTime="1.867829867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:51:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:51:37.053347535 +0000 UTC m=+48.794922830" watchObservedRunningTime="2026-07-16 16:51:37.05466088 +0000 UTC m=+48.796236179" Jul 16 16:51:37.991827 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:37.991792 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bd2d4507-465f-4002-8953-9e289841cdbb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-5tsmr\" (UID: \"bd2d4507-465f-4002-8953-9e289841cdbb\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5tsmr" Jul 16 16:51:37.991996 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:37.991948 2571 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 16:51:37.992062 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:37.992027 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd2d4507-465f-4002-8953-9e289841cdbb-cluster-monitoring-operator-tls podName:bd2d4507-465f-4002-8953-9e289841cdbb nodeName:}" failed. No retries permitted until 2026-07-16 16:51:41.992005392 +0000 UTC m=+53.733580681 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/bd2d4507-465f-4002-8953-9e289841cdbb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-5tsmr" (UID: "bd2d4507-465f-4002-8953-9e289841cdbb") : secret "cluster-monitoring-operator-tls" not found Jul 16 16:51:38.093038 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:38.092984 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d67e5a27-8c54-42e3-af31-f7abfd8ddf95-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-5ctvn\" (UID: \"d67e5a27-8c54-42e3-af31-f7abfd8ddf95\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5ctvn" Jul 16 16:51:38.093463 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:38.093080 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2bdea19-f5df-4297-8304-da19a8cc7a62-service-ca-bundle\") pod \"router-default-57bc6f48c6-kt2wd\" (UID: \"d2bdea19-f5df-4297-8304-da19a8cc7a62\") " pod="openshift-ingress/router-default-57bc6f48c6-kt2wd" Jul 16 16:51:38.093463 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:38.093141 2571 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 16:51:38.093463 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:38.093182 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2bdea19-f5df-4297-8304-da19a8cc7a62-metrics-certs\") pod \"router-default-57bc6f48c6-kt2wd\" (UID: \"d2bdea19-f5df-4297-8304-da19a8cc7a62\") " pod="openshift-ingress/router-default-57bc6f48c6-kt2wd" Jul 16 16:51:38.093463 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:38.093215 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d67e5a27-8c54-42e3-af31-f7abfd8ddf95-samples-operator-tls podName:d67e5a27-8c54-42e3-af31-f7abfd8ddf95 nodeName:}" failed. No retries permitted until 2026-07-16 16:51:42.093194717 +0000 UTC m=+53.834770040 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d67e5a27-8c54-42e3-af31-f7abfd8ddf95-samples-operator-tls") pod "cluster-samples-operator-65db858894-5ctvn" (UID: "d67e5a27-8c54-42e3-af31-f7abfd8ddf95") : secret "samples-operator-tls" not found Jul 16 16:51:38.093463 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:38.093272 2571 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 16:51:38.093463 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:38.093303 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d2bdea19-f5df-4297-8304-da19a8cc7a62-service-ca-bundle podName:d2bdea19-f5df-4297-8304-da19a8cc7a62 nodeName:}" failed. No retries permitted until 2026-07-16 16:51:42.093277973 +0000 UTC m=+53.834853274 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d2bdea19-f5df-4297-8304-da19a8cc7a62-service-ca-bundle") pod "router-default-57bc6f48c6-kt2wd" (UID: "d2bdea19-f5df-4297-8304-da19a8cc7a62") : configmap references non-existent config key: service-ca.crt Jul 16 16:51:38.093463 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:38.093361 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2bdea19-f5df-4297-8304-da19a8cc7a62-metrics-certs podName:d2bdea19-f5df-4297-8304-da19a8cc7a62 nodeName:}" failed. No retries permitted until 2026-07-16 16:51:42.093346805 +0000 UTC m=+53.834922095 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2bdea19-f5df-4297-8304-da19a8cc7a62-metrics-certs") pod "router-default-57bc6f48c6-kt2wd" (UID: "d2bdea19-f5df-4297-8304-da19a8cc7a62") : secret "router-metrics-certs-default" not found Jul 16 16:51:39.404688 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:39.404662 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/77f8326b-54be-49e5-a855-647cd2cd05f8-registry-tls\") pod \"image-registry-5574f576bc-zfnfv\" (UID: \"77f8326b-54be-49e5-a855-647cd2cd05f8\") " pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" Jul 16 16:51:39.404688 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:39.404691 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/959268a1-18d8-4f7b-8c63-ba175ba63fe9-metrics-tls\") pod \"dns-default-2bnf8\" (UID: \"959268a1-18d8-4f7b-8c63-ba175ba63fe9\") " pod="openshift-dns/dns-default-2bnf8" Jul 16 16:51:39.405016 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:39.404835 2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 16:51:39.405016 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:39.404856 2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5574f576bc-zfnfv: secret "image-registry-tls" not found Jul 16 16:51:39.405016 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:39.404842 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 16:51:39.405016 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:39.404906 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/77f8326b-54be-49e5-a855-647cd2cd05f8-registry-tls podName:77f8326b-54be-49e5-a855-647cd2cd05f8 nodeName:}" failed. No retries permitted until 2026-07-16 16:51:55.404891626 +0000 UTC m=+67.146466900 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/77f8326b-54be-49e5-a855-647cd2cd05f8-registry-tls") pod "image-registry-5574f576bc-zfnfv" (UID: "77f8326b-54be-49e5-a855-647cd2cd05f8") : secret "image-registry-tls" not found Jul 16 16:51:39.405016 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:39.404918 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/959268a1-18d8-4f7b-8c63-ba175ba63fe9-metrics-tls podName:959268a1-18d8-4f7b-8c63-ba175ba63fe9 nodeName:}" failed. No retries permitted until 2026-07-16 16:51:55.404912589 +0000 UTC m=+67.146487863 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/959268a1-18d8-4f7b-8c63-ba175ba63fe9-metrics-tls") pod "dns-default-2bnf8" (UID: "959268a1-18d8-4f7b-8c63-ba175ba63fe9") : secret "dns-default-metrics-tls" not found Jul 16 16:51:40.047409 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:40.047375 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-5t9rp" event={"ID":"37bad57e-950d-4a6a-984b-19f240be539b","Type":"ContainerStarted","Data":"e5551e447d95e9c5d78e5daccfccf9c5abda111bc0d4b02499ebc792bad47999"} Jul 16 16:51:40.049551 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:40.049499 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-6zvr8" event={"ID":"3d9162b4-f5a7-4aff-afd6-73383ad17f4a","Type":"ContainerStarted","Data":"f1fc337f18983f145253f656eaa342f2d9de39d9d238af7b623bf2a5a6ddb67c"} Jul 16 16:51:40.051594 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:40.051442 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r8cn4_f637d961-672d-4aa2-a49a-fd021c2d4df2/console-operator/0.log" Jul 16 16:51:40.051594 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:40.051475 2571 generic.go:356] "Generic (PLEG): container finished" podID="f637d961-672d-4aa2-a49a-fd021c2d4df2" containerID="61b144616a00eb93d618f58213065e4ce924596dc1facb2a7b56020aad191f16" exitCode=255 Jul 16 16:51:40.051594 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:40.051546 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-r8cn4" event={"ID":"f637d961-672d-4aa2-a49a-fd021c2d4df2","Type":"ContainerDied","Data":"61b144616a00eb93d618f58213065e4ce924596dc1facb2a7b56020aad191f16"} Jul 16 16:51:40.051837 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:40.051774 2571 scope.go:117] "RemoveContainer" containerID="61b144616a00eb93d618f58213065e4ce924596dc1facb2a7b56020aad191f16" Jul 16 16:51:40.053845 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:40.053817 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fjsn9" event={"ID":"32cd8d7c-6e88-483e-a5d7-f028311b4be3","Type":"ContainerStarted","Data":"f506a962ab583319ec4bbcc2b5f420ec34c9ba538a266d3ef7f4cd75abfedcd1"} Jul 16 16:51:40.066838 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:40.066714 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-5t9rp" podStartSLOduration=1.322395841 podStartE2EDuration="6.066698203s" totalImagesPullingTime="4.744302362s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:51:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:51:40.066300421 +0000 UTC m=+51.807875716" watchObservedRunningTime="2026-07-16 16:51:40.066698203 +0000 UTC m=+51.808273500" Jul 16 16:51:40.104106 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:40.103435 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-6zvr8" podStartSLOduration=1.45316929 podStartE2EDuration="6.103417262s" totalImagesPullingTime="4.650247972s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:51:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:51:40.101910601 +0000 UTC m=+51.843485898" watchObservedRunningTime="2026-07-16 16:51:40.103417262 +0000 UTC m=+51.844992559" Jul 16 16:51:40.121124 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:40.121069 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fjsn9" podStartSLOduration=1.292031707 podStartE2EDuration="6.121052353s" totalImagesPullingTime="4.829020646s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:51:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:51:40.119804414 +0000 UTC m=+51.861379709" watchObservedRunningTime="2026-07-16 16:51:40.121052353 +0000 UTC m=+51.862627651" Jul 16 16:51:41.060531 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:41.060503 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r8cn4_f637d961-672d-4aa2-a49a-fd021c2d4df2/console-operator/1.log" Jul 16 16:51:41.061072 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:41.061052 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r8cn4_f637d961-672d-4aa2-a49a-fd021c2d4df2/console-operator/0.log" Jul 16 16:51:41.061169 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:41.061094 2571 generic.go:356] "Generic (PLEG): container finished" podID="f637d961-672d-4aa2-a49a-fd021c2d4df2" containerID="9ade8f09fcabf062af21c2546346b97d0ab357252101a78f998b74360c45464a" exitCode=255 Jul 16 16:51:41.061224 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:41.061182 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-r8cn4" event={"ID":"f637d961-672d-4aa2-a49a-fd021c2d4df2","Type":"ContainerDied","Data":"9ade8f09fcabf062af21c2546346b97d0ab357252101a78f998b74360c45464a"} Jul 16 16:51:41.061270 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:41.061238 2571 scope.go:117] "RemoveContainer" containerID="61b144616a00eb93d618f58213065e4ce924596dc1facb2a7b56020aad191f16" Jul 16 16:51:41.061514 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:41.061494 2571 scope.go:117] "RemoveContainer" containerID="9ade8f09fcabf062af21c2546346b97d0ab357252101a78f998b74360c45464a" Jul 16 16:51:41.061703 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:41.061678 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-r8cn4_openshift-console-operator(f637d961-672d-4aa2-a49a-fd021c2d4df2)\"" pod="openshift-console-operator/console-operator-7866497494-r8cn4" podUID="f637d961-672d-4aa2-a49a-fd021c2d4df2" Jul 16 16:51:41.658747 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:41.658701 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-cghnz"] Jul 16 16:51:41.698398 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:41.698367 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cghnz"] Jul 16 16:51:41.698540 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:41.698504 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cghnz" Jul 16 16:51:41.701032 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:41.701001 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rbpsf\"" Jul 16 16:51:41.701218 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:41.701193 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 16 16:51:41.701984 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:41.701959 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 16 16:51:41.827995 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:41.827968 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2c867807-c284-400a-a28c-b702c95abd6d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cghnz\" (UID: \"2c867807-c284-400a-a28c-b702c95abd6d\") " pod="openshift-insights/insights-runtime-extractor-cghnz" Jul 16 16:51:41.828167 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:41.828070 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxzqh\" (UniqueName: \"kubernetes.io/projected/2c867807-c284-400a-a28c-b702c95abd6d-kube-api-access-gxzqh\") pod \"insights-runtime-extractor-cghnz\" (UID: \"2c867807-c284-400a-a28c-b702c95abd6d\") " pod="openshift-insights/insights-runtime-extractor-cghnz" Jul 16 16:51:41.828167 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:41.828109 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2c867807-c284-400a-a28c-b702c95abd6d-crio-socket\") pod \"insights-runtime-extractor-cghnz\" (UID: \"2c867807-c284-400a-a28c-b702c95abd6d\") " pod="openshift-insights/insights-runtime-extractor-cghnz" Jul 16 16:51:41.828389 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:41.828356 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2c867807-c284-400a-a28c-b702c95abd6d-data-volume\") pod \"insights-runtime-extractor-cghnz\" (UID: \"2c867807-c284-400a-a28c-b702c95abd6d\") " pod="openshift-insights/insights-runtime-extractor-cghnz" Jul 16 16:51:41.828509 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:41.828455 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2c867807-c284-400a-a28c-b702c95abd6d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cghnz\" (UID: \"2c867807-c284-400a-a28c-b702c95abd6d\") " pod="openshift-insights/insights-runtime-extractor-cghnz" Jul 16 16:51:41.928844 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:41.928809 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2c867807-c284-400a-a28c-b702c95abd6d-data-volume\") pod \"insights-runtime-extractor-cghnz\" (UID: \"2c867807-c284-400a-a28c-b702c95abd6d\") " pod="openshift-insights/insights-runtime-extractor-cghnz" Jul 16 16:51:41.928982 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:41.928901 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2c867807-c284-400a-a28c-b702c95abd6d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cghnz\" (UID: \"2c867807-c284-400a-a28c-b702c95abd6d\") " pod="openshift-insights/insights-runtime-extractor-cghnz" Jul 16 16:51:41.928982 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:41.928941 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2c867807-c284-400a-a28c-b702c95abd6d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cghnz\" (UID: \"2c867807-c284-400a-a28c-b702c95abd6d\") " pod="openshift-insights/insights-runtime-extractor-cghnz" Jul 16 16:51:41.929106 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:41.929027 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxzqh\" (UniqueName: \"kubernetes.io/projected/2c867807-c284-400a-a28c-b702c95abd6d-kube-api-access-gxzqh\") pod \"insights-runtime-extractor-cghnz\" (UID: \"2c867807-c284-400a-a28c-b702c95abd6d\") " pod="openshift-insights/insights-runtime-extractor-cghnz" Jul 16 16:51:41.929106 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:41.929067 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2c867807-c284-400a-a28c-b702c95abd6d-crio-socket\") pod \"insights-runtime-extractor-cghnz\" (UID: \"2c867807-c284-400a-a28c-b702c95abd6d\") " pod="openshift-insights/insights-runtime-extractor-cghnz" Jul 16 16:51:41.929106 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:41.929076 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 16:51:41.929245 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:41.929100 2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 16:51:41.929245 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:41.929155 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c867807-c284-400a-a28c-b702c95abd6d-insights-runtime-extractor-tls podName:2c867807-c284-400a-a28c-b702c95abd6d nodeName:}" failed. No retries permitted until 2026-07-16 16:51:42.429125154 +0000 UTC m=+54.170700449 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2c867807-c284-400a-a28c-b702c95abd6d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cghnz" (UID: "2c867807-c284-400a-a28c-b702c95abd6d") : secret "insights-runtime-extractor-tls" not found Jul 16 16:51:41.929245 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:41.929176 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2c867807-c284-400a-a28c-b702c95abd6d-kube-rbac-proxy-cm podName:2c867807-c284-400a-a28c-b702c95abd6d nodeName:}" failed. No retries permitted until 2026-07-16 16:51:42.429165537 +0000 UTC m=+54.170740824 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2c867807-c284-400a-a28c-b702c95abd6d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cghnz" (UID: "2c867807-c284-400a-a28c-b702c95abd6d") : configmap "kube-rbac-proxy" not found Jul 16 16:51:41.929245 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:41.929193 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2c867807-c284-400a-a28c-b702c95abd6d-data-volume\") pod \"insights-runtime-extractor-cghnz\" (UID: \"2c867807-c284-400a-a28c-b702c95abd6d\") " pod="openshift-insights/insights-runtime-extractor-cghnz" Jul 16 16:51:41.929429 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:41.929263 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2c867807-c284-400a-a28c-b702c95abd6d-crio-socket\") pod \"insights-runtime-extractor-cghnz\" (UID: \"2c867807-c284-400a-a28c-b702c95abd6d\") " pod="openshift-insights/insights-runtime-extractor-cghnz" Jul 16 16:51:41.940931 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:41.940888 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxzqh\" (UniqueName: \"kubernetes.io/projected/2c867807-c284-400a-a28c-b702c95abd6d-kube-api-access-gxzqh\") pod \"insights-runtime-extractor-cghnz\" (UID: \"2c867807-c284-400a-a28c-b702c95abd6d\") " pod="openshift-insights/insights-runtime-extractor-cghnz" Jul 16 16:51:42.029794 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:42.029767 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bd2d4507-465f-4002-8953-9e289841cdbb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-5tsmr\" (UID: \"bd2d4507-465f-4002-8953-9e289841cdbb\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5tsmr" Jul 16 16:51:42.029917 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:42.029900 2571 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 16:51:42.029967 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:42.029957 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd2d4507-465f-4002-8953-9e289841cdbb-cluster-monitoring-operator-tls podName:bd2d4507-465f-4002-8953-9e289841cdbb nodeName:}" failed. No retries permitted until 2026-07-16 16:51:50.029942846 +0000 UTC m=+61.771518121 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/bd2d4507-465f-4002-8953-9e289841cdbb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-5tsmr" (UID: "bd2d4507-465f-4002-8953-9e289841cdbb") : secret "cluster-monitoring-operator-tls" not found Jul 16 16:51:42.063737 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:42.063714 2571 scope.go:117] "RemoveContainer" containerID="9ade8f09fcabf062af21c2546346b97d0ab357252101a78f998b74360c45464a" Jul 16 16:51:42.064013 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:42.063856 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-r8cn4_openshift-console-operator(f637d961-672d-4aa2-a49a-fd021c2d4df2)\"" pod="openshift-console-operator/console-operator-7866497494-r8cn4" podUID="f637d961-672d-4aa2-a49a-fd021c2d4df2" Jul 16 16:51:42.130878 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:42.130829 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2bdea19-f5df-4297-8304-da19a8cc7a62-metrics-certs\") pod \"router-default-57bc6f48c6-kt2wd\" (UID: \"d2bdea19-f5df-4297-8304-da19a8cc7a62\") " pod="openshift-ingress/router-default-57bc6f48c6-kt2wd" Jul 16 16:51:42.130972 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:42.130935 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d67e5a27-8c54-42e3-af31-f7abfd8ddf95-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-5ctvn\" (UID: \"d67e5a27-8c54-42e3-af31-f7abfd8ddf95\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5ctvn" Jul 16 16:51:42.131049 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:42.131030 2571 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 16:51:42.131099 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:42.131060 2571 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 16:51:42.131099 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:42.131078 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2bdea19-f5df-4297-8304-da19a8cc7a62-service-ca-bundle\") pod \"router-default-57bc6f48c6-kt2wd\" (UID: \"d2bdea19-f5df-4297-8304-da19a8cc7a62\") " pod="openshift-ingress/router-default-57bc6f48c6-kt2wd" Jul 16 16:51:42.131219 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:42.131098 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2bdea19-f5df-4297-8304-da19a8cc7a62-metrics-certs podName:d2bdea19-f5df-4297-8304-da19a8cc7a62 nodeName:}" failed. No retries permitted until 2026-07-16 16:51:50.131077561 +0000 UTC m=+61.872652854 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2bdea19-f5df-4297-8304-da19a8cc7a62-metrics-certs") pod "router-default-57bc6f48c6-kt2wd" (UID: "d2bdea19-f5df-4297-8304-da19a8cc7a62") : secret "router-metrics-certs-default" not found Jul 16 16:51:42.131219 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:42.131148 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d67e5a27-8c54-42e3-af31-f7abfd8ddf95-samples-operator-tls podName:d67e5a27-8c54-42e3-af31-f7abfd8ddf95 nodeName:}" failed. No retries permitted until 2026-07-16 16:51:50.131134477 +0000 UTC m=+61.872709753 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d67e5a27-8c54-42e3-af31-f7abfd8ddf95-samples-operator-tls") pod "cluster-samples-operator-65db858894-5ctvn" (UID: "d67e5a27-8c54-42e3-af31-f7abfd8ddf95") : secret "samples-operator-tls" not found Jul 16 16:51:42.131219 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:42.131182 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d2bdea19-f5df-4297-8304-da19a8cc7a62-service-ca-bundle podName:d2bdea19-f5df-4297-8304-da19a8cc7a62 nodeName:}" failed. No retries permitted until 2026-07-16 16:51:50.131158783 +0000 UTC m=+61.872734067 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d2bdea19-f5df-4297-8304-da19a8cc7a62-service-ca-bundle") pod "router-default-57bc6f48c6-kt2wd" (UID: "d2bdea19-f5df-4297-8304-da19a8cc7a62") : configmap references non-existent config key: service-ca.crt Jul 16 16:51:42.433286 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:42.433266 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2c867807-c284-400a-a28c-b702c95abd6d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cghnz\" (UID: \"2c867807-c284-400a-a28c-b702c95abd6d\") " pod="openshift-insights/insights-runtime-extractor-cghnz" Jul 16 16:51:42.433377 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:42.433301 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2c867807-c284-400a-a28c-b702c95abd6d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cghnz\" (UID: \"2c867807-c284-400a-a28c-b702c95abd6d\") " pod="openshift-insights/insights-runtime-extractor-cghnz" Jul 16 16:51:42.433434 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:42.433410 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 16:51:42.433481 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:42.433435 2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 16:51:42.433481 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:42.433470 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c867807-c284-400a-a28c-b702c95abd6d-insights-runtime-extractor-tls podName:2c867807-c284-400a-a28c-b702c95abd6d nodeName:}" failed. No retries permitted until 2026-07-16 16:51:43.433451295 +0000 UTC m=+55.175026587 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2c867807-c284-400a-a28c-b702c95abd6d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cghnz" (UID: "2c867807-c284-400a-a28c-b702c95abd6d") : secret "insights-runtime-extractor-tls" not found Jul 16 16:51:42.433583 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:42.433490 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2c867807-c284-400a-a28c-b702c95abd6d-kube-rbac-proxy-cm podName:2c867807-c284-400a-a28c-b702c95abd6d nodeName:}" failed. No retries permitted until 2026-07-16 16:51:43.433478859 +0000 UTC m=+55.175054140 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2c867807-c284-400a-a28c-b702c95abd6d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cghnz" (UID: "2c867807-c284-400a-a28c-b702c95abd6d") : configmap "kube-rbac-proxy" not found Jul 16 16:51:43.068043 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:43.068020 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r8cn4_f637d961-672d-4aa2-a49a-fd021c2d4df2/console-operator/1.log" Jul 16 16:51:43.069602 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:43.069571 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-lbhfp" event={"ID":"ae213c51-b545-4cf5-a244-0f10ff891fab","Type":"ContainerStarted","Data":"39e0d07afe22ec9e8a3ea8def4c7d4b08e1d4738357ba9bb6c1087fa1c2b2528"} Jul 16 16:51:43.117305 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:43.117268 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-lbhfp" podStartSLOduration=1.999182123 podStartE2EDuration="8.117254824s" totalImagesPullingTime="6.118072701s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:51:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:51:43.115846329 +0000 UTC m=+54.857421630" watchObservedRunningTime="2026-07-16 16:51:43.117254824 +0000 UTC m=+54.858830142" Jul 16 16:51:43.440975 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:43.440952 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2c867807-c284-400a-a28c-b702c95abd6d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cghnz\" (UID: \"2c867807-c284-400a-a28c-b702c95abd6d\") " pod="openshift-insights/insights-runtime-extractor-cghnz" Jul 16 16:51:43.441074 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:43.440987 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2c867807-c284-400a-a28c-b702c95abd6d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cghnz\" (UID: \"2c867807-c284-400a-a28c-b702c95abd6d\") " pod="openshift-insights/insights-runtime-extractor-cghnz" Jul 16 16:51:43.441136 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:43.441118 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 16:51:43.441180 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:43.441142 2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 16:51:43.441224 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:43.441192 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c867807-c284-400a-a28c-b702c95abd6d-insights-runtime-extractor-tls podName:2c867807-c284-400a-a28c-b702c95abd6d nodeName:}" failed. No retries permitted until 2026-07-16 16:51:45.441173896 +0000 UTC m=+57.182749174 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2c867807-c284-400a-a28c-b702c95abd6d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cghnz" (UID: "2c867807-c284-400a-a28c-b702c95abd6d") : secret "insights-runtime-extractor-tls" not found Jul 16 16:51:43.441224 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:43.441210 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2c867807-c284-400a-a28c-b702c95abd6d-kube-rbac-proxy-cm podName:2c867807-c284-400a-a28c-b702c95abd6d nodeName:}" failed. No retries permitted until 2026-07-16 16:51:45.441200269 +0000 UTC m=+57.182775545 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2c867807-c284-400a-a28c-b702c95abd6d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cghnz" (UID: "2c867807-c284-400a-a28c-b702c95abd6d") : configmap "kube-rbac-proxy" not found Jul 16 16:51:44.150767 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:44.150740 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fttnf_f7e16013-94ca-4a61-b551-f3f3b66d3cb1/dns-node-resolver/0.log" Jul 16 16:51:44.712571 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:44.712548 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-r8cn4" Jul 16 16:51:44.712695 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:44.712621 2571 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-r8cn4" Jul 16 16:51:44.712938 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:44.712926 2571 scope.go:117] "RemoveContainer" containerID="9ade8f09fcabf062af21c2546346b97d0ab357252101a78f998b74360c45464a" Jul 16 16:51:44.713072 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:44.713055 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-r8cn4_openshift-console-operator(f637d961-672d-4aa2-a49a-fd021c2d4df2)\"" pod="openshift-console-operator/console-operator-7866497494-r8cn4" podUID="f637d961-672d-4aa2-a49a-fd021c2d4df2" Jul 16 16:51:45.144329 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:45.144271 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ln8q5_1dd02f2a-342f-46a3-bbba-b630378fa3db/node-ca/0.log" Jul 16 16:51:45.457338 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:45.457308 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2c867807-c284-400a-a28c-b702c95abd6d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cghnz\" (UID: \"2c867807-c284-400a-a28c-b702c95abd6d\") " pod="openshift-insights/insights-runtime-extractor-cghnz" Jul 16 16:51:45.457763 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:45.457353 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2c867807-c284-400a-a28c-b702c95abd6d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cghnz\" (UID: \"2c867807-c284-400a-a28c-b702c95abd6d\") " pod="openshift-insights/insights-runtime-extractor-cghnz" Jul 16 16:51:45.457763 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:45.457458 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 16:51:45.457763 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:45.457502 2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 16:51:45.457763 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:45.457527 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c867807-c284-400a-a28c-b702c95abd6d-insights-runtime-extractor-tls podName:2c867807-c284-400a-a28c-b702c95abd6d nodeName:}" failed. No retries permitted until 2026-07-16 16:51:49.457509892 +0000 UTC m=+61.199085183 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2c867807-c284-400a-a28c-b702c95abd6d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cghnz" (UID: "2c867807-c284-400a-a28c-b702c95abd6d") : secret "insights-runtime-extractor-tls" not found Jul 16 16:51:45.457763 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:45.457563 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2c867807-c284-400a-a28c-b702c95abd6d-kube-rbac-proxy-cm podName:2c867807-c284-400a-a28c-b702c95abd6d nodeName:}" failed. No retries permitted until 2026-07-16 16:51:49.457546186 +0000 UTC m=+61.199121460 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2c867807-c284-400a-a28c-b702c95abd6d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cghnz" (UID: "2c867807-c284-400a-a28c-b702c95abd6d") : configmap "kube-rbac-proxy" not found Jul 16 16:51:45.945263 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:45.945234 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xxcr4_d8306746-2174-40cd-9319-7137be8a4f5e/serve-healthcheck-canary/0.log" Jul 16 16:51:46.546649 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:46.546619 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-fjsn9_32cd8d7c-6e88-483e-a5d7-f028311b4be3/kube-storage-version-migrator-operator/0.log" Jul 16 16:51:49.491372 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:49.491322 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2c867807-c284-400a-a28c-b702c95abd6d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cghnz\" (UID: \"2c867807-c284-400a-a28c-b702c95abd6d\") " pod="openshift-insights/insights-runtime-extractor-cghnz" Jul 16 16:51:49.491842 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:49.491379 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2c867807-c284-400a-a28c-b702c95abd6d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cghnz\" (UID: \"2c867807-c284-400a-a28c-b702c95abd6d\") " pod="openshift-insights/insights-runtime-extractor-cghnz" Jul 16 16:51:49.491842 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:49.491499 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 16:51:49.491842 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:49.491518 2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 16:51:49.491842 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:49.491581 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2c867807-c284-400a-a28c-b702c95abd6d-kube-rbac-proxy-cm podName:2c867807-c284-400a-a28c-b702c95abd6d nodeName:}" failed. No retries permitted until 2026-07-16 16:51:57.491560278 +0000 UTC m=+69.233135562 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2c867807-c284-400a-a28c-b702c95abd6d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cghnz" (UID: "2c867807-c284-400a-a28c-b702c95abd6d") : configmap "kube-rbac-proxy" not found Jul 16 16:51:49.491842 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:49.491596 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c867807-c284-400a-a28c-b702c95abd6d-insights-runtime-extractor-tls podName:2c867807-c284-400a-a28c-b702c95abd6d nodeName:}" failed. No retries permitted until 2026-07-16 16:51:57.491589436 +0000 UTC m=+69.233164710 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2c867807-c284-400a-a28c-b702c95abd6d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cghnz" (UID: "2c867807-c284-400a-a28c-b702c95abd6d") : secret "insights-runtime-extractor-tls" not found Jul 16 16:51:49.978334 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:49.978301 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qd7bg" Jul 16 16:51:50.096870 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:50.096840 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bd2d4507-465f-4002-8953-9e289841cdbb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-5tsmr\" (UID: \"bd2d4507-465f-4002-8953-9e289841cdbb\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5tsmr" Jul 16 16:51:50.097097 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:50.097077 2571 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 16:51:50.097193 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:50.097147 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd2d4507-465f-4002-8953-9e289841cdbb-cluster-monitoring-operator-tls podName:bd2d4507-465f-4002-8953-9e289841cdbb nodeName:}" failed. No retries permitted until 2026-07-16 16:52:06.09712679 +0000 UTC m=+77.838702065 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/bd2d4507-465f-4002-8953-9e289841cdbb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-5tsmr" (UID: "bd2d4507-465f-4002-8953-9e289841cdbb") : secret "cluster-monitoring-operator-tls" not found Jul 16 16:51:50.197996 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:50.197962 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2bdea19-f5df-4297-8304-da19a8cc7a62-metrics-certs\") pod \"router-default-57bc6f48c6-kt2wd\" (UID: \"d2bdea19-f5df-4297-8304-da19a8cc7a62\") " pod="openshift-ingress/router-default-57bc6f48c6-kt2wd" Jul 16 16:51:50.198138 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:50.198036 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d67e5a27-8c54-42e3-af31-f7abfd8ddf95-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-5ctvn\" (UID: \"d67e5a27-8c54-42e3-af31-f7abfd8ddf95\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5ctvn" Jul 16 16:51:50.198138 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:50.198089 2571 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 16:51:50.198138 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:50.198107 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2bdea19-f5df-4297-8304-da19a8cc7a62-service-ca-bundle\") pod \"router-default-57bc6f48c6-kt2wd\" (UID: \"d2bdea19-f5df-4297-8304-da19a8cc7a62\") " pod="openshift-ingress/router-default-57bc6f48c6-kt2wd" Jul 16 16:51:50.198138 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:50.198138 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2bdea19-f5df-4297-8304-da19a8cc7a62-metrics-certs podName:d2bdea19-f5df-4297-8304-da19a8cc7a62 nodeName:}" failed. No retries permitted until 2026-07-16 16:52:06.19812522 +0000 UTC m=+77.939700494 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2bdea19-f5df-4297-8304-da19a8cc7a62-metrics-certs") pod "router-default-57bc6f48c6-kt2wd" (UID: "d2bdea19-f5df-4297-8304-da19a8cc7a62") : secret "router-metrics-certs-default" not found Jul 16 16:51:50.198336 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:50.198254 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d2bdea19-f5df-4297-8304-da19a8cc7a62-service-ca-bundle podName:d2bdea19-f5df-4297-8304-da19a8cc7a62 nodeName:}" failed. No retries permitted until 2026-07-16 16:52:06.198236572 +0000 UTC m=+77.939811851 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d2bdea19-f5df-4297-8304-da19a8cc7a62-service-ca-bundle") pod "router-default-57bc6f48c6-kt2wd" (UID: "d2bdea19-f5df-4297-8304-da19a8cc7a62") : configmap references non-existent config key: service-ca.crt Jul 16 16:51:50.200429 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:50.200396 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d67e5a27-8c54-42e3-af31-f7abfd8ddf95-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-5ctvn\" (UID: \"d67e5a27-8c54-42e3-af31-f7abfd8ddf95\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5ctvn" Jul 16 16:51:50.274928 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:50.274870 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-wz9sj\"" Jul 16 16:51:50.282420 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:50.282401 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5ctvn" Jul 16 16:51:50.411539 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:50.411510 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5ctvn"] Jul 16 16:51:51.096350 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:51.096313 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5ctvn" event={"ID":"d67e5a27-8c54-42e3-af31-f7abfd8ddf95","Type":"ContainerStarted","Data":"9434cf20e8ae2ad5c47193a88b337a0149e3b50789b24cf5117b610a6407bb0c"} Jul 16 16:51:52.106589 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:52.106560 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5ctvn" event={"ID":"d67e5a27-8c54-42e3-af31-f7abfd8ddf95","Type":"ContainerStarted","Data":"50ebc34593e329124b2abc4f82a579aa86afe0758aa73e111380714662e17138"} Jul 16 16:51:53.114106 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:53.114069 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5ctvn" event={"ID":"d67e5a27-8c54-42e3-af31-f7abfd8ddf95","Type":"ContainerStarted","Data":"fbc05f88ff9bfbf0efe3d4ecb369e6d53562568ca200979e98156cf567825dae"} Jul 16 16:51:53.131432 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:53.131387 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5ctvn" podStartSLOduration=17.581943909 podStartE2EDuration="19.131374368s" totalImagesPullingTime="1.549430459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:51:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:51:53.130761032 +0000 UTC m=+64.872336326" watchObservedRunningTime="2026-07-16 16:51:53.131374368 +0000 UTC m=+64.872949664" Jul 16 16:51:54.535684 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:54.535647 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2341bdcc-01b6-426c-8189-71be7fdc2e8b-metrics-certs\") pod \"network-metrics-daemon-4p6nc\" (UID: \"2341bdcc-01b6-426c-8189-71be7fdc2e8b\") " pod="openshift-multus/network-metrics-daemon-4p6nc" Jul 16 16:51:54.538069 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:54.538044 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 16:51:54.546086 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:54.546069 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 16 16:51:54.546135 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:54.546124 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2341bdcc-01b6-426c-8189-71be7fdc2e8b-metrics-certs podName:2341bdcc-01b6-426c-8189-71be7fdc2e8b nodeName:}" failed. No retries permitted until 2026-07-16 16:52:58.546108063 +0000 UTC m=+130.287683338 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2341bdcc-01b6-426c-8189-71be7fdc2e8b-metrics-certs") pod "network-metrics-daemon-4p6nc" (UID: "2341bdcc-01b6-426c-8189-71be7fdc2e8b") : secret "metrics-daemon-secret" not found Jul 16 16:51:54.636561 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:54.636522 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/598a7966-d69c-4867-9936-667f0c47ed23-original-pull-secret\") pod \"global-pull-secret-syncer-wd65t\" (UID: \"598a7966-d69c-4867-9936-667f0c47ed23\") " pod="kube-system/global-pull-secret-syncer-wd65t" Jul 16 16:51:54.640254 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:54.640232 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 16:51:54.649797 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:54.649769 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/598a7966-d69c-4867-9936-667f0c47ed23-original-pull-secret\") pod \"global-pull-secret-syncer-wd65t\" (UID: \"598a7966-d69c-4867-9936-667f0c47ed23\") " pod="kube-system/global-pull-secret-syncer-wd65t" Jul 16 16:51:54.709893 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:54.709865 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wd65t" Jul 16 16:51:54.737316 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:54.737287 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx468\" (UniqueName: \"kubernetes.io/projected/1c20e16c-0039-4705-a02f-d2a8a5c66ae1-kube-api-access-hx468\") pod \"network-check-target-zfpzk\" (UID: \"1c20e16c-0039-4705-a02f-d2a8a5c66ae1\") " pod="openshift-network-diagnostics/network-check-target-zfpzk" Jul 16 16:51:54.739563 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:54.739541 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx468\" (UniqueName: \"kubernetes.io/projected/1c20e16c-0039-4705-a02f-d2a8a5c66ae1-kube-api-access-hx468\") pod \"network-check-target-zfpzk\" (UID: \"1c20e16c-0039-4705-a02f-d2a8a5c66ae1\") " pod="openshift-network-diagnostics/network-check-target-zfpzk" Jul 16 16:51:54.839246 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:54.839220 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wd65t"] Jul 16 16:51:54.842057 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:51:54.842027 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod598a7966_d69c_4867_9936_667f0c47ed23.slice/crio-542bab5283642bbcaaddff0df675478708f33bf1ad41299231e9303dc33e95d1 WatchSource:0}: Error finding container 542bab5283642bbcaaddff0df675478708f33bf1ad41299231e9303dc33e95d1: Status 404 returned error can't find the container with id 542bab5283642bbcaaddff0df675478708f33bf1ad41299231e9303dc33e95d1 Jul 16 16:51:54.987560 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:54.987535 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-h8r6p\"" Jul 16 16:51:54.995797 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:54.995773 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zfpzk" Jul 16 16:51:55.120915 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:55.120882 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wd65t" event={"ID":"598a7966-d69c-4867-9936-667f0c47ed23","Type":"ContainerStarted","Data":"542bab5283642bbcaaddff0df675478708f33bf1ad41299231e9303dc33e95d1"} Jul 16 16:51:55.128055 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:55.128033 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zfpzk"] Jul 16 16:51:55.131005 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:51:55.130980 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c20e16c_0039_4705_a02f_d2a8a5c66ae1.slice/crio-5742cdbaf73d5d8bf7c61eb90291f9fc87b3737fd5af6a31c27ce469dbbec401 WatchSource:0}: Error finding container 5742cdbaf73d5d8bf7c61eb90291f9fc87b3737fd5af6a31c27ce469dbbec401: Status 404 returned error can't find the container with id 5742cdbaf73d5d8bf7c61eb90291f9fc87b3737fd5af6a31c27ce469dbbec401 Jul 16 16:51:55.444424 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:55.444398 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/77f8326b-54be-49e5-a855-647cd2cd05f8-registry-tls\") pod \"image-registry-5574f576bc-zfnfv\" (UID: \"77f8326b-54be-49e5-a855-647cd2cd05f8\") " pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" Jul 16 16:51:55.444424 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:55.444426 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/959268a1-18d8-4f7b-8c63-ba175ba63fe9-metrics-tls\") pod \"dns-default-2bnf8\" (UID: \"959268a1-18d8-4f7b-8c63-ba175ba63fe9\") " pod="openshift-dns/dns-default-2bnf8" Jul 16 16:51:55.446710 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:55.446684 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/77f8326b-54be-49e5-a855-647cd2cd05f8-registry-tls\") pod \"image-registry-5574f576bc-zfnfv\" (UID: \"77f8326b-54be-49e5-a855-647cd2cd05f8\") " pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" Jul 16 16:51:55.446891 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:55.446870 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/959268a1-18d8-4f7b-8c63-ba175ba63fe9-metrics-tls\") pod \"dns-default-2bnf8\" (UID: \"959268a1-18d8-4f7b-8c63-ba175ba63fe9\") " pod="openshift-dns/dns-default-2bnf8" Jul 16 16:51:55.719804 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:55.719740 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2wxqg\"" Jul 16 16:51:55.727597 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:55.727573 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" Jul 16 16:51:55.741686 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:55.741655 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xtf2c\"" Jul 16 16:51:55.749249 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:55.749224 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2bnf8" Jul 16 16:51:55.772611 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:55.772590 2571 scope.go:117] "RemoveContainer" containerID="9ade8f09fcabf062af21c2546346b97d0ab357252101a78f998b74360c45464a" Jul 16 16:51:55.880949 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:55.880883 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5574f576bc-zfnfv"] Jul 16 16:51:55.885463 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:51:55.885430 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77f8326b_54be_49e5_a855_647cd2cd05f8.slice/crio-51c2e5ffe80ba0ac7c11d67519947b4e37bd9c26b565ada9f40911f2275034f5 WatchSource:0}: Error finding container 51c2e5ffe80ba0ac7c11d67519947b4e37bd9c26b565ada9f40911f2275034f5: Status 404 returned error can't find the container with id 51c2e5ffe80ba0ac7c11d67519947b4e37bd9c26b565ada9f40911f2275034f5 Jul 16 16:51:55.904344 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:55.904315 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2bnf8"] Jul 16 16:51:55.920043 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:51:55.920016 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod959268a1_18d8_4f7b_8c63_ba175ba63fe9.slice/crio-e83779488949f8fe5ec14f558c1b313ea86cd11986898f62830faa65a9f52ac9 WatchSource:0}: Error finding container e83779488949f8fe5ec14f558c1b313ea86cd11986898f62830faa65a9f52ac9: Status 404 returned error can't find the container with id e83779488949f8fe5ec14f558c1b313ea86cd11986898f62830faa65a9f52ac9 Jul 16 16:51:56.125806 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:56.125773 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2bnf8" event={"ID":"959268a1-18d8-4f7b-8c63-ba175ba63fe9","Type":"ContainerStarted","Data":"e83779488949f8fe5ec14f558c1b313ea86cd11986898f62830faa65a9f52ac9"} Jul 16 16:51:56.127803 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:56.127532 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" event={"ID":"77f8326b-54be-49e5-a855-647cd2cd05f8","Type":"ContainerStarted","Data":"0b3a77772590cec77d6236db8f4f0998fcaa3615ff6ffd7d0e9634e3aef4f5be"} Jul 16 16:51:56.127803 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:56.127578 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" event={"ID":"77f8326b-54be-49e5-a855-647cd2cd05f8","Type":"ContainerStarted","Data":"51c2e5ffe80ba0ac7c11d67519947b4e37bd9c26b565ada9f40911f2275034f5"} Jul 16 16:51:56.127803 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:56.127664 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" Jul 16 16:51:56.129636 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:56.129611 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zfpzk" event={"ID":"1c20e16c-0039-4705-a02f-d2a8a5c66ae1","Type":"ContainerStarted","Data":"b48377060d6db408233c603534b0973dd6f2f674b52d4249774f6473ffec9e33"} Jul 16 16:51:56.129789 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:56.129651 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zfpzk" event={"ID":"1c20e16c-0039-4705-a02f-d2a8a5c66ae1","Type":"ContainerStarted","Data":"5742cdbaf73d5d8bf7c61eb90291f9fc87b3737fd5af6a31c27ce469dbbec401"} Jul 16 16:51:56.129789 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:56.129764 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-zfpzk" Jul 16 16:51:56.131329 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:56.131310 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r8cn4_f637d961-672d-4aa2-a49a-fd021c2d4df2/console-operator/2.log" Jul 16 16:51:56.131793 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:56.131772 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r8cn4_f637d961-672d-4aa2-a49a-fd021c2d4df2/console-operator/1.log" Jul 16 16:51:56.131935 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:56.131810 2571 generic.go:356] "Generic (PLEG): container finished" podID="f637d961-672d-4aa2-a49a-fd021c2d4df2" containerID="6ff36fac7777eb2de37ebe4eb61bbfe5eb82b81b0b05a0897823f46d76648f21" exitCode=255 Jul 16 16:51:56.131935 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:56.131840 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-r8cn4" event={"ID":"f637d961-672d-4aa2-a49a-fd021c2d4df2","Type":"ContainerDied","Data":"6ff36fac7777eb2de37ebe4eb61bbfe5eb82b81b0b05a0897823f46d76648f21"} Jul 16 16:51:56.131935 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:56.131883 2571 scope.go:117] "RemoveContainer" containerID="9ade8f09fcabf062af21c2546346b97d0ab357252101a78f998b74360c45464a" Jul 16 16:51:56.132180 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:56.132161 2571 scope.go:117] "RemoveContainer" containerID="6ff36fac7777eb2de37ebe4eb61bbfe5eb82b81b0b05a0897823f46d76648f21" Jul 16 16:51:56.132356 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:56.132334 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-r8cn4_openshift-console-operator(f637d961-672d-4aa2-a49a-fd021c2d4df2)\"" pod="openshift-console-operator/console-operator-7866497494-r8cn4" podUID="f637d961-672d-4aa2-a49a-fd021c2d4df2" Jul 16 16:51:56.153395 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:56.153350 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" podStartSLOduration=67.153334848 podStartE2EDuration="1m7.153334848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:50:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:51:56.151480711 +0000 UTC m=+67.893056007" watchObservedRunningTime="2026-07-16 16:51:56.153334848 +0000 UTC m=+67.894910145" Jul 16 16:51:56.188738 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:56.188684 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-zfpzk" podStartSLOduration=68.18866835 podStartE2EDuration="1m8.18866835s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:50:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:51:56.170046184 +0000 UTC m=+67.911621481" watchObservedRunningTime="2026-07-16 16:51:56.18866835 +0000 UTC m=+67.930243647" Jul 16 16:51:57.138555 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:57.138522 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r8cn4_f637d961-672d-4aa2-a49a-fd021c2d4df2/console-operator/2.log" Jul 16 16:51:57.563099 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:57.563066 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2c867807-c284-400a-a28c-b702c95abd6d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cghnz\" (UID: \"2c867807-c284-400a-a28c-b702c95abd6d\") " pod="openshift-insights/insights-runtime-extractor-cghnz" Jul 16 16:51:57.563295 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:57.563123 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2c867807-c284-400a-a28c-b702c95abd6d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cghnz\" (UID: \"2c867807-c284-400a-a28c-b702c95abd6d\") " pod="openshift-insights/insights-runtime-extractor-cghnz" Jul 16 16:51:57.563295 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:57.563241 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 16:51:57.563399 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:51:57.563324 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c867807-c284-400a-a28c-b702c95abd6d-insights-runtime-extractor-tls podName:2c867807-c284-400a-a28c-b702c95abd6d nodeName:}" failed. No retries permitted until 2026-07-16 16:52:13.563302194 +0000 UTC m=+85.304877470 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2c867807-c284-400a-a28c-b702c95abd6d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cghnz" (UID: "2c867807-c284-400a-a28c-b702c95abd6d") : secret "insights-runtime-extractor-tls" not found Jul 16 16:51:57.563690 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:51:57.563664 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2c867807-c284-400a-a28c-b702c95abd6d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cghnz\" (UID: \"2c867807-c284-400a-a28c-b702c95abd6d\") " pod="openshift-insights/insights-runtime-extractor-cghnz" Jul 16 16:52:00.152114 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:00.152028 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wd65t" event={"ID":"598a7966-d69c-4867-9936-667f0c47ed23","Type":"ContainerStarted","Data":"b86626ba6aec2eec88467bcc02cd091eeb001c7929774d719ad36a3de4b0acbe"} Jul 16 16:52:00.153759 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:00.153738 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2bnf8" event={"ID":"959268a1-18d8-4f7b-8c63-ba175ba63fe9","Type":"ContainerStarted","Data":"5d9d54b417a1b568e56e285f0b600c9439fcb290af41f608b54c7246dace9030"} Jul 16 16:52:00.153851 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:00.153774 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2bnf8" event={"ID":"959268a1-18d8-4f7b-8c63-ba175ba63fe9","Type":"ContainerStarted","Data":"f6f0362f96f0d09f5759fe440f237f6ab455f2b6a5facdbbc2cc544a7befface"} Jul 16 16:52:00.153851 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:00.153812 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2bnf8" Jul 16 16:52:00.171825 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:00.171785 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-wd65t" podStartSLOduration=66.518657897 podStartE2EDuration="1m11.171771246s" totalImagesPullingTime="4.653113349s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 16:50:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:52:00.171679483 +0000 UTC m=+71.913254778" watchObservedRunningTime="2026-07-16 16:52:00.171771246 +0000 UTC m=+71.913346543" Jul 16 16:52:00.194476 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:00.194416 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2bnf8" podStartSLOduration=33.633017554 podStartE2EDuration="37.19440356s" totalImagesPullingTime="3.561386006s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:51:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:52:00.193323202 +0000 UTC m=+71.934898508" watchObservedRunningTime="2026-07-16 16:52:00.19440356 +0000 UTC m=+71.935978892" Jul 16 16:52:04.712225 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:04.712191 2571 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-r8cn4" Jul 16 16:52:04.712663 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:04.712242 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-r8cn4" Jul 16 16:52:04.712663 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:04.712491 2571 scope.go:117] "RemoveContainer" containerID="6ff36fac7777eb2de37ebe4eb61bbfe5eb82b81b0b05a0897823f46d76648f21" Jul 16 16:52:04.712663 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:52:04.712615 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-r8cn4_openshift-console-operator(f637d961-672d-4aa2-a49a-fd021c2d4df2)\"" pod="openshift-console-operator/console-operator-7866497494-r8cn4" podUID="f637d961-672d-4aa2-a49a-fd021c2d4df2" Jul 16 16:52:06.125119 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:06.125068 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bd2d4507-465f-4002-8953-9e289841cdbb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-5tsmr\" (UID: \"bd2d4507-465f-4002-8953-9e289841cdbb\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5tsmr" Jul 16 16:52:06.127456 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:06.127434 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bd2d4507-465f-4002-8953-9e289841cdbb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-5tsmr\" (UID: \"bd2d4507-465f-4002-8953-9e289841cdbb\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5tsmr" Jul 16 16:52:06.225954 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:06.225920 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2bdea19-f5df-4297-8304-da19a8cc7a62-service-ca-bundle\") pod \"router-default-57bc6f48c6-kt2wd\" (UID: \"d2bdea19-f5df-4297-8304-da19a8cc7a62\") " pod="openshift-ingress/router-default-57bc6f48c6-kt2wd" Jul 16 16:52:06.226081 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:06.225990 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2bdea19-f5df-4297-8304-da19a8cc7a62-metrics-certs\") pod \"router-default-57bc6f48c6-kt2wd\" (UID: \"d2bdea19-f5df-4297-8304-da19a8cc7a62\") " pod="openshift-ingress/router-default-57bc6f48c6-kt2wd" Jul 16 16:52:06.226468 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:06.226446 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2bdea19-f5df-4297-8304-da19a8cc7a62-service-ca-bundle\") pod \"router-default-57bc6f48c6-kt2wd\" (UID: \"d2bdea19-f5df-4297-8304-da19a8cc7a62\") " pod="openshift-ingress/router-default-57bc6f48c6-kt2wd" Jul 16 16:52:06.228074 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:06.228050 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2bdea19-f5df-4297-8304-da19a8cc7a62-metrics-certs\") pod \"router-default-57bc6f48c6-kt2wd\" (UID: \"d2bdea19-f5df-4297-8304-da19a8cc7a62\") " pod="openshift-ingress/router-default-57bc6f48c6-kt2wd" Jul 16 16:52:06.377822 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:06.377752 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-2qdtq\"" Jul 16 16:52:06.386060 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:06.386045 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5tsmr" Jul 16 16:52:06.500499 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:06.500476 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-8q47g\"" Jul 16 16:52:06.507339 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:06.507320 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5tsmr"] Jul 16 16:52:06.509062 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:06.509046 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-57bc6f48c6-kt2wd" Jul 16 16:52:06.509627 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:52:06.509606 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd2d4507_465f_4002_8953_9e289841cdbb.slice/crio-7319a9b476e54bee99ead5f96ebbb5ec92efe2ecce8e18c16448f9f491c50a9b WatchSource:0}: Error finding container 7319a9b476e54bee99ead5f96ebbb5ec92efe2ecce8e18c16448f9f491c50a9b: Status 404 returned error can't find the container with id 7319a9b476e54bee99ead5f96ebbb5ec92efe2ecce8e18c16448f9f491c50a9b Jul 16 16:52:06.631826 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:06.631752 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-57bc6f48c6-kt2wd"] Jul 16 16:52:06.634683 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:52:06.634657 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2bdea19_f5df_4297_8304_da19a8cc7a62.slice/crio-a6a47760c91aae4b2db42e71593ce09801fa969614c57e66c34c1f2074ce3fca WatchSource:0}: Error finding container a6a47760c91aae4b2db42e71593ce09801fa969614c57e66c34c1f2074ce3fca: Status 404 returned error can't find the container with id a6a47760c91aae4b2db42e71593ce09801fa969614c57e66c34c1f2074ce3fca Jul 16 16:52:07.176937 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:07.176898 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-57bc6f48c6-kt2wd" event={"ID":"d2bdea19-f5df-4297-8304-da19a8cc7a62","Type":"ContainerStarted","Data":"144d23f66df68ad7f45539341ce5253f9575b3da5272adf2be46200a3ecafbf1"} Jul 16 16:52:07.177395 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:07.176957 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-57bc6f48c6-kt2wd" event={"ID":"d2bdea19-f5df-4297-8304-da19a8cc7a62","Type":"ContainerStarted","Data":"a6a47760c91aae4b2db42e71593ce09801fa969614c57e66c34c1f2074ce3fca"} Jul 16 16:52:07.178192 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:07.178156 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5tsmr" event={"ID":"bd2d4507-465f-4002-8953-9e289841cdbb","Type":"ContainerStarted","Data":"7319a9b476e54bee99ead5f96ebbb5ec92efe2ecce8e18c16448f9f491c50a9b"} Jul 16 16:52:07.200244 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:07.200196 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-57bc6f48c6-kt2wd" podStartSLOduration=33.200182151 podStartE2EDuration="33.200182151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:51:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:52:07.198790428 +0000 UTC m=+78.940365727" watchObservedRunningTime="2026-07-16 16:52:07.200182151 +0000 UTC m=+78.941757451" Jul 16 16:52:07.510124 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:07.510044 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-57bc6f48c6-kt2wd" Jul 16 16:52:07.512561 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:07.512539 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-57bc6f48c6-kt2wd" Jul 16 16:52:07.782122 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:07.782041 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-64c7758767-r6jhx"] Jul 16 16:52:07.786134 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:07.786111 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64c7758767-r6jhx" Jul 16 16:52:07.790695 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:07.790677 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 16 16:52:07.791931 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:07.791853 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 16 16:52:07.791931 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:07.791856 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 16 16:52:07.791931 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:07.791903 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 16 16:52:07.798799 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:07.798781 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-64c7758767-r6jhx"] Jul 16 16:52:07.837084 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:07.836883 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98kzw\" (UniqueName: \"kubernetes.io/projected/baf3d239-4179-4cae-8f2a-73d9ec5fc7d0-kube-api-access-98kzw\") pod \"klusterlet-addon-workmgr-64c7758767-r6jhx\" (UID: \"baf3d239-4179-4cae-8f2a-73d9ec5fc7d0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64c7758767-r6jhx" Jul 16 16:52:07.837084 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:07.837017 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/baf3d239-4179-4cae-8f2a-73d9ec5fc7d0-tmp\") pod \"klusterlet-addon-workmgr-64c7758767-r6jhx\" (UID: \"baf3d239-4179-4cae-8f2a-73d9ec5fc7d0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64c7758767-r6jhx" Jul 16 16:52:07.837084 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:07.837047 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/baf3d239-4179-4cae-8f2a-73d9ec5fc7d0-klusterlet-config\") pod \"klusterlet-addon-workmgr-64c7758767-r6jhx\" (UID: \"baf3d239-4179-4cae-8f2a-73d9ec5fc7d0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64c7758767-r6jhx" Jul 16 16:52:07.938293 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:07.938261 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/baf3d239-4179-4cae-8f2a-73d9ec5fc7d0-tmp\") pod \"klusterlet-addon-workmgr-64c7758767-r6jhx\" (UID: \"baf3d239-4179-4cae-8f2a-73d9ec5fc7d0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64c7758767-r6jhx" Jul 16 16:52:07.938293 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:07.938295 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/baf3d239-4179-4cae-8f2a-73d9ec5fc7d0-klusterlet-config\") pod \"klusterlet-addon-workmgr-64c7758767-r6jhx\" (UID: \"baf3d239-4179-4cae-8f2a-73d9ec5fc7d0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64c7758767-r6jhx" Jul 16 16:52:07.938501 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:07.938324 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98kzw\" (UniqueName: \"kubernetes.io/projected/baf3d239-4179-4cae-8f2a-73d9ec5fc7d0-kube-api-access-98kzw\") pod \"klusterlet-addon-workmgr-64c7758767-r6jhx\" (UID: \"baf3d239-4179-4cae-8f2a-73d9ec5fc7d0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64c7758767-r6jhx" Jul 16 16:52:07.938722 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:07.938696 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/baf3d239-4179-4cae-8f2a-73d9ec5fc7d0-tmp\") pod \"klusterlet-addon-workmgr-64c7758767-r6jhx\" (UID: \"baf3d239-4179-4cae-8f2a-73d9ec5fc7d0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64c7758767-r6jhx" Jul 16 16:52:07.940888 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:07.940866 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/baf3d239-4179-4cae-8f2a-73d9ec5fc7d0-klusterlet-config\") pod \"klusterlet-addon-workmgr-64c7758767-r6jhx\" (UID: \"baf3d239-4179-4cae-8f2a-73d9ec5fc7d0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64c7758767-r6jhx" Jul 16 16:52:07.970367 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:07.970334 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98kzw\" (UniqueName: \"kubernetes.io/projected/baf3d239-4179-4cae-8f2a-73d9ec5fc7d0-kube-api-access-98kzw\") pod \"klusterlet-addon-workmgr-64c7758767-r6jhx\" (UID: \"baf3d239-4179-4cae-8f2a-73d9ec5fc7d0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64c7758767-r6jhx" Jul 16 16:52:08.097833 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:08.097753 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64c7758767-r6jhx" Jul 16 16:52:08.182526 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:08.182491 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-57bc6f48c6-kt2wd" Jul 16 16:52:08.183933 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:08.183913 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-57bc6f48c6-kt2wd" Jul 16 16:52:08.500101 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:08.500078 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-64c7758767-r6jhx"] Jul 16 16:52:08.502376 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:52:08.502353 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaf3d239_4179_4cae_8f2a_73d9ec5fc7d0.slice/crio-2f80366af55ced30530607e4c88a1cd65c2f4461838644186f0f6744d3cef0ab WatchSource:0}: Error finding container 2f80366af55ced30530607e4c88a1cd65c2f4461838644186f0f6744d3cef0ab: Status 404 returned error can't find the container with id 2f80366af55ced30530607e4c88a1cd65c2f4461838644186f0f6744d3cef0ab Jul 16 16:52:09.188300 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:09.188223 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64c7758767-r6jhx" event={"ID":"baf3d239-4179-4cae-8f2a-73d9ec5fc7d0","Type":"ContainerStarted","Data":"2f80366af55ced30530607e4c88a1cd65c2f4461838644186f0f6744d3cef0ab"} Jul 16 16:52:09.190356 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:09.190286 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5tsmr" event={"ID":"bd2d4507-465f-4002-8953-9e289841cdbb","Type":"ContainerStarted","Data":"36e62dbdbc262eb41a87bdba87402b589bc2e349bc3e3334964af9680b3fadad"} Jul 16 16:52:09.209502 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:09.209441 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5tsmr" podStartSLOduration=33.299527684 podStartE2EDuration="35.209426546s" totalImagesPullingTime="1.909898862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:51:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:52:09.207929108 +0000 UTC m=+80.949504404" watchObservedRunningTime="2026-07-16 16:52:09.209426546 +0000 UTC m=+80.951001840" Jul 16 16:52:10.159994 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:10.159960 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2bnf8" Jul 16 16:52:12.205088 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:12.205054 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64c7758767-r6jhx" event={"ID":"baf3d239-4179-4cae-8f2a-73d9ec5fc7d0","Type":"ContainerStarted","Data":"5c176d3ee5dad71f5372da562be7dbdabb46702c9712d3c10c013a0a7e33e917"} Jul 16 16:52:12.205509 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:12.205272 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64c7758767-r6jhx" Jul 16 16:52:12.206648 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:12.206629 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64c7758767-r6jhx" Jul 16 16:52:12.227537 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:12.227497 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64c7758767-r6jhx" podStartSLOduration=1.6515127170000001 podStartE2EDuration="5.227483653s" totalImagesPullingTime="3.575970936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:52:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:52:12.226638716 +0000 UTC m=+83.968214012" watchObservedRunningTime="2026-07-16 16:52:12.227483653 +0000 UTC m=+83.969058948" Jul 16 16:52:13.586172 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:13.586134 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2c867807-c284-400a-a28c-b702c95abd6d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cghnz\" (UID: \"2c867807-c284-400a-a28c-b702c95abd6d\") " pod="openshift-insights/insights-runtime-extractor-cghnz" Jul 16 16:52:13.588395 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:13.588378 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2c867807-c284-400a-a28c-b702c95abd6d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cghnz\" (UID: \"2c867807-c284-400a-a28c-b702c95abd6d\") " pod="openshift-insights/insights-runtime-extractor-cghnz" Jul 16 16:52:13.812342 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:13.812309 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rbpsf\"" Jul 16 16:52:13.820528 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:13.820509 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cghnz" Jul 16 16:52:13.972555 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:13.972368 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cghnz"] Jul 16 16:52:14.212694 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:14.212657 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cghnz" event={"ID":"2c867807-c284-400a-a28c-b702c95abd6d","Type":"ContainerStarted","Data":"0f9863515505c163268b0e0b5534c470da2e994f87f9387997101d16fb4f26f8"} Jul 16 16:52:15.733129 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:15.733092 2571 patch_prober.go:28] interesting pod/image-registry-5574f576bc-zfnfv container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 16 16:52:15.733660 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:15.733153 2571 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" podUID="77f8326b-54be-49e5-a855-647cd2cd05f8" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 16:52:16.771536 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:16.771503 2571 scope.go:117] "RemoveContainer" containerID="6ff36fac7777eb2de37ebe4eb61bbfe5eb82b81b0b05a0897823f46d76648f21" Jul 16 16:52:17.143689 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:17.143617 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" Jul 16 16:52:19.233875 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:19.233836 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cghnz" event={"ID":"2c867807-c284-400a-a28c-b702c95abd6d","Type":"ContainerStarted","Data":"6b754529f055c2bbe9c8002eb737e303d55b3b26c5a44dfb2330a99ac7ef5c46"} Jul 16 16:52:19.235982 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:19.235963 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r8cn4_f637d961-672d-4aa2-a49a-fd021c2d4df2/console-operator/2.log" Jul 16 16:52:19.236095 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:19.236008 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-r8cn4" event={"ID":"f637d961-672d-4aa2-a49a-fd021c2d4df2","Type":"ContainerStarted","Data":"6d70e8c5a9347e8aa9461e221f3d51f25feb60f2ec75042234651f7025106486"} Jul 16 16:52:19.236525 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:19.236503 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-r8cn4" Jul 16 16:52:19.241012 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:19.240996 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-r8cn4" Jul 16 16:52:19.259832 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:19.259793 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-r8cn4" podStartSLOduration=40.559495295 podStartE2EDuration="45.259781152s" totalImagesPullingTime="4.700285857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:51:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:52:19.259005036 +0000 UTC m=+91.000580331" watchObservedRunningTime="2026-07-16 16:52:19.259781152 +0000 UTC m=+91.001356447" Jul 16 16:52:23.253322 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:23.253282 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cghnz" event={"ID":"2c867807-c284-400a-a28c-b702c95abd6d","Type":"ContainerStarted","Data":"02c262b5bce5e01d6cc2bf1acf15332650643dd9583c8f78943b7f7a023498dc"} Jul 16 16:52:26.265555 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:26.265521 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cghnz" event={"ID":"2c867807-c284-400a-a28c-b702c95abd6d","Type":"ContainerStarted","Data":"527d1329d0491a49ba246db2745bbb268c681c87c0ff47816bd6545b39139232"} Jul 16 16:52:26.287405 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:26.287356 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-cghnz" podStartSLOduration=33.471660022 podStartE2EDuration="45.287339921s" totalImagesPullingTime="11.815679899s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 16:51:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:52:26.286318355 +0000 UTC m=+98.027893650" watchObservedRunningTime="2026-07-16 16:52:26.287339921 +0000 UTC m=+98.028915217" Jul 16 16:52:27.141208 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:27.141156 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-zfpzk" Jul 16 16:52:28.469338 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:28.469300 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-fmz26"] Jul 16 16:52:28.473908 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:28.473892 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fmz26" Jul 16 16:52:28.476611 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:28.476590 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 16 16:52:28.476750 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:28.476631 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 16 16:52:28.476750 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:28.476711 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-2tl75\"" Jul 16 16:52:28.477820 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:28.477803 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 16 16:52:28.478007 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:28.477994 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 16 16:52:28.514780 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:28.514757 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9gcs\" (UniqueName: \"kubernetes.io/projected/fdbc551c-5bfd-4f74-8050-523205d01489-kube-api-access-l9gcs\") pod \"node-exporter-fmz26\" (UID: \"fdbc551c-5bfd-4f74-8050-523205d01489\") " pod="openshift-monitoring/node-exporter-fmz26" Jul 16 16:52:28.514870 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:28.514790 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fdbc551c-5bfd-4f74-8050-523205d01489-metrics-client-ca\") pod \"node-exporter-fmz26\" (UID: \"fdbc551c-5bfd-4f74-8050-523205d01489\") " pod="openshift-monitoring/node-exporter-fmz26" Jul 16 16:52:28.514870 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:28.514809 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fdbc551c-5bfd-4f74-8050-523205d01489-node-exporter-accelerators-collector-config\") pod \"node-exporter-fmz26\" (UID: \"fdbc551c-5bfd-4f74-8050-523205d01489\") " pod="openshift-monitoring/node-exporter-fmz26" Jul 16 16:52:28.514870 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:28.514841 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fdbc551c-5bfd-4f74-8050-523205d01489-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fmz26\" (UID: \"fdbc551c-5bfd-4f74-8050-523205d01489\") " pod="openshift-monitoring/node-exporter-fmz26" Jul 16 16:52:28.514870 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:28.514856 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fdbc551c-5bfd-4f74-8050-523205d01489-root\") pod \"node-exporter-fmz26\" (UID: \"fdbc551c-5bfd-4f74-8050-523205d01489\") " pod="openshift-monitoring/node-exporter-fmz26" Jul 16 16:52:28.514870 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:28.514869 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fdbc551c-5bfd-4f74-8050-523205d01489-node-exporter-textfile\") pod \"node-exporter-fmz26\" (UID: \"fdbc551c-5bfd-4f74-8050-523205d01489\") " pod="openshift-monitoring/node-exporter-fmz26" Jul 16 16:52:28.515065 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:28.514890 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fdbc551c-5bfd-4f74-8050-523205d01489-sys\") pod \"node-exporter-fmz26\" (UID: \"fdbc551c-5bfd-4f74-8050-523205d01489\") " pod="openshift-monitoring/node-exporter-fmz26" Jul 16 16:52:28.515065 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:28.514959 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fdbc551c-5bfd-4f74-8050-523205d01489-node-exporter-tls\") pod \"node-exporter-fmz26\" (UID: \"fdbc551c-5bfd-4f74-8050-523205d01489\") " pod="openshift-monitoring/node-exporter-fmz26" Jul 16 16:52:28.515065 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:28.514981 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fdbc551c-5bfd-4f74-8050-523205d01489-node-exporter-wtmp\") pod \"node-exporter-fmz26\" (UID: \"fdbc551c-5bfd-4f74-8050-523205d01489\") " pod="openshift-monitoring/node-exporter-fmz26" Jul 16 16:52:28.615804 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:28.615782 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9gcs\" (UniqueName: \"kubernetes.io/projected/fdbc551c-5bfd-4f74-8050-523205d01489-kube-api-access-l9gcs\") pod \"node-exporter-fmz26\" (UID: \"fdbc551c-5bfd-4f74-8050-523205d01489\") " pod="openshift-monitoring/node-exporter-fmz26" Jul 16 16:52:28.615914 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:28.615814 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fdbc551c-5bfd-4f74-8050-523205d01489-metrics-client-ca\") pod \"node-exporter-fmz26\" (UID: \"fdbc551c-5bfd-4f74-8050-523205d01489\") " pod="openshift-monitoring/node-exporter-fmz26" Jul 16 16:52:28.615914 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:28.615831 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fdbc551c-5bfd-4f74-8050-523205d01489-node-exporter-accelerators-collector-config\") pod \"node-exporter-fmz26\" (UID: \"fdbc551c-5bfd-4f74-8050-523205d01489\") " pod="openshift-monitoring/node-exporter-fmz26" Jul 16 16:52:28.615914 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:28.615855 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fdbc551c-5bfd-4f74-8050-523205d01489-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fmz26\" (UID: \"fdbc551c-5bfd-4f74-8050-523205d01489\") " pod="openshift-monitoring/node-exporter-fmz26" Jul 16 16:52:28.615914 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:28.615872 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fdbc551c-5bfd-4f74-8050-523205d01489-root\") pod \"node-exporter-fmz26\" (UID: \"fdbc551c-5bfd-4f74-8050-523205d01489\") " pod="openshift-monitoring/node-exporter-fmz26" Jul 16 16:52:28.615914 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:28.615890 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fdbc551c-5bfd-4f74-8050-523205d01489-node-exporter-textfile\") pod \"node-exporter-fmz26\" (UID: \"fdbc551c-5bfd-4f74-8050-523205d01489\") " pod="openshift-monitoring/node-exporter-fmz26" Jul 16 16:52:28.616370 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:28.615970 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fdbc551c-5bfd-4f74-8050-523205d01489-root\") pod \"node-exporter-fmz26\" (UID: \"fdbc551c-5bfd-4f74-8050-523205d01489\") " pod="openshift-monitoring/node-exporter-fmz26" Jul 16 16:52:28.616370 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:28.616018 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fdbc551c-5bfd-4f74-8050-523205d01489-sys\") pod \"node-exporter-fmz26\" (UID: \"fdbc551c-5bfd-4f74-8050-523205d01489\") " pod="openshift-monitoring/node-exporter-fmz26" Jul 16 16:52:28.616370 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:28.616090 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fdbc551c-5bfd-4f74-8050-523205d01489-node-exporter-tls\") pod \"node-exporter-fmz26\" (UID: \"fdbc551c-5bfd-4f74-8050-523205d01489\") " pod="openshift-monitoring/node-exporter-fmz26" Jul 16 16:52:28.616370 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:28.616102 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fdbc551c-5bfd-4f74-8050-523205d01489-sys\") pod \"node-exporter-fmz26\" (UID: \"fdbc551c-5bfd-4f74-8050-523205d01489\") " pod="openshift-monitoring/node-exporter-fmz26" Jul 16 16:52:28.616370 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:28.616111 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fdbc551c-5bfd-4f74-8050-523205d01489-node-exporter-wtmp\") pod \"node-exporter-fmz26\" (UID: \"fdbc551c-5bfd-4f74-8050-523205d01489\") " pod="openshift-monitoring/node-exporter-fmz26" Jul 16 16:52:28.616370 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:28.616244 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fdbc551c-5bfd-4f74-8050-523205d01489-node-exporter-wtmp\") pod \"node-exporter-fmz26\" (UID: \"fdbc551c-5bfd-4f74-8050-523205d01489\") " pod="openshift-monitoring/node-exporter-fmz26" Jul 16 16:52:28.616370 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:28.616286 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fdbc551c-5bfd-4f74-8050-523205d01489-node-exporter-textfile\") pod \"node-exporter-fmz26\" (UID: \"fdbc551c-5bfd-4f74-8050-523205d01489\") " pod="openshift-monitoring/node-exporter-fmz26" Jul 16 16:52:28.616662 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:28.616456 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fdbc551c-5bfd-4f74-8050-523205d01489-node-exporter-accelerators-collector-config\") pod \"node-exporter-fmz26\" (UID: \"fdbc551c-5bfd-4f74-8050-523205d01489\") " pod="openshift-monitoring/node-exporter-fmz26" Jul 16 16:52:28.616662 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:28.616499 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fdbc551c-5bfd-4f74-8050-523205d01489-metrics-client-ca\") pod \"node-exporter-fmz26\" (UID: \"fdbc551c-5bfd-4f74-8050-523205d01489\") " pod="openshift-monitoring/node-exporter-fmz26" Jul 16 16:52:28.618170 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:28.618153 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fdbc551c-5bfd-4f74-8050-523205d01489-node-exporter-tls\") pod \"node-exporter-fmz26\" (UID: \"fdbc551c-5bfd-4f74-8050-523205d01489\") " pod="openshift-monitoring/node-exporter-fmz26" Jul 16 16:52:28.618296 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:28.618280 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fdbc551c-5bfd-4f74-8050-523205d01489-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fmz26\" (UID: \"fdbc551c-5bfd-4f74-8050-523205d01489\") " pod="openshift-monitoring/node-exporter-fmz26" Jul 16 16:52:28.624573 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:28.624554 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9gcs\" (UniqueName: \"kubernetes.io/projected/fdbc551c-5bfd-4f74-8050-523205d01489-kube-api-access-l9gcs\") pod \"node-exporter-fmz26\" (UID: \"fdbc551c-5bfd-4f74-8050-523205d01489\") " pod="openshift-monitoring/node-exporter-fmz26" Jul 16 16:52:28.782514 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:28.782437 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fmz26" Jul 16 16:52:28.793970 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:52:28.793942 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdbc551c_5bfd_4f74_8050_523205d01489.slice/crio-4aedf2abad2171f004ef6d6b5efbd23a3f486d33b94bf189499dfddd718049f9 WatchSource:0}: Error finding container 4aedf2abad2171f004ef6d6b5efbd23a3f486d33b94bf189499dfddd718049f9: Status 404 returned error can't find the container with id 4aedf2abad2171f004ef6d6b5efbd23a3f486d33b94bf189499dfddd718049f9 Jul 16 16:52:29.276777 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:29.276723 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fmz26" event={"ID":"fdbc551c-5bfd-4f74-8050-523205d01489","Type":"ContainerStarted","Data":"4aedf2abad2171f004ef6d6b5efbd23a3f486d33b94bf189499dfddd718049f9"} Jul 16 16:52:30.281666 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:30.281632 2571 generic.go:356] "Generic (PLEG): container finished" podID="fdbc551c-5bfd-4f74-8050-523205d01489" containerID="ef31db8747e4b1ad6f2f8de54c611ee13a73bf07263451fc286c224bbd5b216f" exitCode=0 Jul 16 16:52:30.282077 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:30.281715 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fmz26" event={"ID":"fdbc551c-5bfd-4f74-8050-523205d01489","Type":"ContainerDied","Data":"ef31db8747e4b1ad6f2f8de54c611ee13a73bf07263451fc286c224bbd5b216f"} Jul 16 16:52:30.313193 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:30.313167 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5574f576bc-zfnfv"] Jul 16 16:52:31.287399 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:31.287368 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fmz26" event={"ID":"fdbc551c-5bfd-4f74-8050-523205d01489","Type":"ContainerStarted","Data":"fa71364fcc85188b187555bb70c7c81d1d1d5b853c551714cf09ba0eb12781e2"} Jul 16 16:52:31.287774 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:31.287417 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fmz26" event={"ID":"fdbc551c-5bfd-4f74-8050-523205d01489","Type":"ContainerStarted","Data":"d09c9bcc4304c036474137cf2e38f0edbad65fe6ce56a0dc725c1a3ae729c0ed"} Jul 16 16:52:31.312127 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:31.312084 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-fmz26" podStartSLOduration=2.300910084 podStartE2EDuration="3.312069921s" totalImagesPullingTime="950.147932ms" totalInitContainerRuntime="61.011905ms" isStatefulPod=true podCreationTimestamp="2026-07-16 16:52:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:52:31.311699714 +0000 UTC m=+103.053275010" watchObservedRunningTime="2026-07-16 16:52:31.312069921 +0000 UTC m=+103.053645217" Jul 16 16:52:40.981412 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:40.981379 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-c758bf489-xhhwx"] Jul 16 16:52:40.985595 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:40.985571 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c758bf489-xhhwx" Jul 16 16:52:40.988402 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:40.988378 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 16 16:52:40.988532 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:40.988427 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 16 16:52:40.988532 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:40.988520 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 16 16:52:40.989822 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:40.989763 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 16 16:52:40.989822 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:40.989786 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 16 16:52:40.989822 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:40.989797 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 16 16:52:40.989822 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:40.989832 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 16 16:52:40.989822 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:40.989833 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-hfqgv\"" Jul 16 16:52:40.994307 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:40.994274 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 16 16:52:40.996125 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:40.996106 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c758bf489-xhhwx"] Jul 16 16:52:41.111444 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:41.111424 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1edbc8b0-0e10-4256-af6f-b8556c599efa-trusted-ca-bundle\") pod \"console-c758bf489-xhhwx\" (UID: \"1edbc8b0-0e10-4256-af6f-b8556c599efa\") " pod="openshift-console/console-c758bf489-xhhwx" Jul 16 16:52:41.111552 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:41.111451 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1edbc8b0-0e10-4256-af6f-b8556c599efa-console-oauth-config\") pod \"console-c758bf489-xhhwx\" (UID: \"1edbc8b0-0e10-4256-af6f-b8556c599efa\") " pod="openshift-console/console-c758bf489-xhhwx" Jul 16 16:52:41.111552 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:41.111490 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1edbc8b0-0e10-4256-af6f-b8556c599efa-console-serving-cert\") pod \"console-c758bf489-xhhwx\" (UID: \"1edbc8b0-0e10-4256-af6f-b8556c599efa\") " pod="openshift-console/console-c758bf489-xhhwx" Jul 16 16:52:41.111622 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:41.111561 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1edbc8b0-0e10-4256-af6f-b8556c599efa-oauth-serving-cert\") pod \"console-c758bf489-xhhwx\" (UID: \"1edbc8b0-0e10-4256-af6f-b8556c599efa\") " pod="openshift-console/console-c758bf489-xhhwx" Jul 16 16:52:41.111622 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:41.111585 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8gjs\" (UniqueName: \"kubernetes.io/projected/1edbc8b0-0e10-4256-af6f-b8556c599efa-kube-api-access-j8gjs\") pod \"console-c758bf489-xhhwx\" (UID: \"1edbc8b0-0e10-4256-af6f-b8556c599efa\") " pod="openshift-console/console-c758bf489-xhhwx" Jul 16 16:52:41.111683 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:41.111617 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1edbc8b0-0e10-4256-af6f-b8556c599efa-console-config\") pod \"console-c758bf489-xhhwx\" (UID: \"1edbc8b0-0e10-4256-af6f-b8556c599efa\") " pod="openshift-console/console-c758bf489-xhhwx" Jul 16 16:52:41.111683 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:41.111639 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1edbc8b0-0e10-4256-af6f-b8556c599efa-service-ca\") pod \"console-c758bf489-xhhwx\" (UID: \"1edbc8b0-0e10-4256-af6f-b8556c599efa\") " pod="openshift-console/console-c758bf489-xhhwx" Jul 16 16:52:41.212550 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:41.212529 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1edbc8b0-0e10-4256-af6f-b8556c599efa-console-serving-cert\") pod \"console-c758bf489-xhhwx\" (UID: \"1edbc8b0-0e10-4256-af6f-b8556c599efa\") " pod="openshift-console/console-c758bf489-xhhwx" Jul 16 16:52:41.212667 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:41.212561 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1edbc8b0-0e10-4256-af6f-b8556c599efa-oauth-serving-cert\") pod \"console-c758bf489-xhhwx\" (UID: \"1edbc8b0-0e10-4256-af6f-b8556c599efa\") " pod="openshift-console/console-c758bf489-xhhwx" Jul 16 16:52:41.212667 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:41.212578 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8gjs\" (UniqueName: \"kubernetes.io/projected/1edbc8b0-0e10-4256-af6f-b8556c599efa-kube-api-access-j8gjs\") pod \"console-c758bf489-xhhwx\" (UID: \"1edbc8b0-0e10-4256-af6f-b8556c599efa\") " pod="openshift-console/console-c758bf489-xhhwx" Jul 16 16:52:41.212667 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:41.212601 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1edbc8b0-0e10-4256-af6f-b8556c599efa-console-config\") pod \"console-c758bf489-xhhwx\" (UID: \"1edbc8b0-0e10-4256-af6f-b8556c599efa\") " pod="openshift-console/console-c758bf489-xhhwx" Jul 16 16:52:41.212667 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:41.212617 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1edbc8b0-0e10-4256-af6f-b8556c599efa-service-ca\") pod \"console-c758bf489-xhhwx\" (UID: \"1edbc8b0-0e10-4256-af6f-b8556c599efa\") " pod="openshift-console/console-c758bf489-xhhwx" Jul 16 16:52:41.212667 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:41.212654 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1edbc8b0-0e10-4256-af6f-b8556c599efa-trusted-ca-bundle\") pod \"console-c758bf489-xhhwx\" (UID: \"1edbc8b0-0e10-4256-af6f-b8556c599efa\") " pod="openshift-console/console-c758bf489-xhhwx" Jul 16 16:52:41.212943 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:41.212671 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1edbc8b0-0e10-4256-af6f-b8556c599efa-console-oauth-config\") pod \"console-c758bf489-xhhwx\" (UID: \"1edbc8b0-0e10-4256-af6f-b8556c599efa\") " pod="openshift-console/console-c758bf489-xhhwx" Jul 16 16:52:41.213660 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:41.213632 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1edbc8b0-0e10-4256-af6f-b8556c599efa-oauth-serving-cert\") pod \"console-c758bf489-xhhwx\" (UID: \"1edbc8b0-0e10-4256-af6f-b8556c599efa\") " pod="openshift-console/console-c758bf489-xhhwx" Jul 16 16:52:41.213857 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:41.213834 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1edbc8b0-0e10-4256-af6f-b8556c599efa-console-config\") pod \"console-c758bf489-xhhwx\" (UID: \"1edbc8b0-0e10-4256-af6f-b8556c599efa\") " pod="openshift-console/console-c758bf489-xhhwx" Jul 16 16:52:41.218055 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:41.214007 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1edbc8b0-0e10-4256-af6f-b8556c599efa-service-ca\") pod \"console-c758bf489-xhhwx\" (UID: \"1edbc8b0-0e10-4256-af6f-b8556c599efa\") " pod="openshift-console/console-c758bf489-xhhwx" Jul 16 16:52:41.218055 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:41.217951 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1edbc8b0-0e10-4256-af6f-b8556c599efa-trusted-ca-bundle\") pod \"console-c758bf489-xhhwx\" (UID: \"1edbc8b0-0e10-4256-af6f-b8556c599efa\") " pod="openshift-console/console-c758bf489-xhhwx" Jul 16 16:52:41.218633 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:41.218615 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1edbc8b0-0e10-4256-af6f-b8556c599efa-console-serving-cert\") pod \"console-c758bf489-xhhwx\" (UID: \"1edbc8b0-0e10-4256-af6f-b8556c599efa\") " pod="openshift-console/console-c758bf489-xhhwx" Jul 16 16:52:41.218716 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:41.218697 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1edbc8b0-0e10-4256-af6f-b8556c599efa-console-oauth-config\") pod \"console-c758bf489-xhhwx\" (UID: \"1edbc8b0-0e10-4256-af6f-b8556c599efa\") " pod="openshift-console/console-c758bf489-xhhwx" Jul 16 16:52:41.221810 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:41.221792 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8gjs\" (UniqueName: \"kubernetes.io/projected/1edbc8b0-0e10-4256-af6f-b8556c599efa-kube-api-access-j8gjs\") pod \"console-c758bf489-xhhwx\" (UID: \"1edbc8b0-0e10-4256-af6f-b8556c599efa\") " pod="openshift-console/console-c758bf489-xhhwx" Jul 16 16:52:41.294651 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:41.294602 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c758bf489-xhhwx" Jul 16 16:52:41.444573 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:41.444541 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c758bf489-xhhwx"] Jul 16 16:52:41.447266 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:52:41.447231 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1edbc8b0_0e10_4256_af6f_b8556c599efa.slice/crio-897441209c4201ce1a50804b8e8dd154fd5debd55676d8656ea1d8803e833987 WatchSource:0}: Error finding container 897441209c4201ce1a50804b8e8dd154fd5debd55676d8656ea1d8803e833987: Status 404 returned error can't find the container with id 897441209c4201ce1a50804b8e8dd154fd5debd55676d8656ea1d8803e833987 Jul 16 16:52:42.326695 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:42.326658 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c758bf489-xhhwx" event={"ID":"1edbc8b0-0e10-4256-af6f-b8556c599efa","Type":"ContainerStarted","Data":"897441209c4201ce1a50804b8e8dd154fd5debd55676d8656ea1d8803e833987"} Jul 16 16:52:44.335807 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:44.335766 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c758bf489-xhhwx" event={"ID":"1edbc8b0-0e10-4256-af6f-b8556c599efa","Type":"ContainerStarted","Data":"80311e91b90b3919ffa6672bdc544ff0dac85b37e0f7dbe97ce9ddfca0e073fa"} Jul 16 16:52:44.373516 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:44.373471 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-c758bf489-xhhwx" podStartSLOduration=1.905381213 podStartE2EDuration="4.373457178s" totalImagesPullingTime="2.468075965s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:52:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:52:44.371641841 +0000 UTC m=+116.113217136" watchObservedRunningTime="2026-07-16 16:52:44.373457178 +0000 UTC m=+116.115032504" Jul 16 16:52:51.295506 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:51.295469 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-c758bf489-xhhwx" Jul 16 16:52:51.295881 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:51.295528 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c758bf489-xhhwx" Jul 16 16:52:51.300252 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:51.300227 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-c758bf489-xhhwx" Jul 16 16:52:51.367608 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:51.367582 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-c758bf489-xhhwx" Jul 16 16:52:55.333597 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:55.333561 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" podUID="77f8326b-54be-49e5-a855-647cd2cd05f8" containerName="registry" containerID="cri-o://0b3a77772590cec77d6236db8f4f0998fcaa3615ff6ffd7d0e9634e3aef4f5be" gracePeriod=30 Jul 16 16:52:55.378493 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:55.378466 2571 generic.go:356] "Generic (PLEG): container finished" podID="32cd8d7c-6e88-483e-a5d7-f028311b4be3" containerID="f506a962ab583319ec4bbcc2b5f420ec34c9ba538a266d3ef7f4cd75abfedcd1" exitCode=0 Jul 16 16:52:55.378620 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:55.378538 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fjsn9" event={"ID":"32cd8d7c-6e88-483e-a5d7-f028311b4be3","Type":"ContainerDied","Data":"f506a962ab583319ec4bbcc2b5f420ec34c9ba538a266d3ef7f4cd75abfedcd1"} Jul 16 16:52:55.378866 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:55.378854 2571 scope.go:117] "RemoveContainer" containerID="f506a962ab583319ec4bbcc2b5f420ec34c9ba538a266d3ef7f4cd75abfedcd1" Jul 16 16:52:55.587851 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:55.587820 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" Jul 16 16:52:55.728018 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:55.727979 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/77f8326b-54be-49e5-a855-647cd2cd05f8-image-registry-private-configuration\") pod \"77f8326b-54be-49e5-a855-647cd2cd05f8\" (UID: \"77f8326b-54be-49e5-a855-647cd2cd05f8\") " Jul 16 16:52:55.728198 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:55.728034 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/77f8326b-54be-49e5-a855-647cd2cd05f8-registry-certificates\") pod \"77f8326b-54be-49e5-a855-647cd2cd05f8\" (UID: \"77f8326b-54be-49e5-a855-647cd2cd05f8\") " Jul 16 16:52:55.728198 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:55.728071 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/77f8326b-54be-49e5-a855-647cd2cd05f8-trusted-ca\") pod \"77f8326b-54be-49e5-a855-647cd2cd05f8\" (UID: \"77f8326b-54be-49e5-a855-647cd2cd05f8\") " Jul 16 16:52:55.728198 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:55.728118 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/77f8326b-54be-49e5-a855-647cd2cd05f8-registry-tls\") pod \"77f8326b-54be-49e5-a855-647cd2cd05f8\" (UID: \"77f8326b-54be-49e5-a855-647cd2cd05f8\") " Jul 16 16:52:55.728198 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:55.728156 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/77f8326b-54be-49e5-a855-647cd2cd05f8-ca-trust-extracted\") pod \"77f8326b-54be-49e5-a855-647cd2cd05f8\" (UID: \"77f8326b-54be-49e5-a855-647cd2cd05f8\") " Jul 16 16:52:55.728198 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:55.728189 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hwxx4\" (UniqueName: \"kubernetes.io/projected/77f8326b-54be-49e5-a855-647cd2cd05f8-kube-api-access-hwxx4\") pod \"77f8326b-54be-49e5-a855-647cd2cd05f8\" (UID: \"77f8326b-54be-49e5-a855-647cd2cd05f8\") " Jul 16 16:52:55.728447 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:55.728247 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/77f8326b-54be-49e5-a855-647cd2cd05f8-bound-sa-token\") pod \"77f8326b-54be-49e5-a855-647cd2cd05f8\" (UID: \"77f8326b-54be-49e5-a855-647cd2cd05f8\") " Jul 16 16:52:55.728447 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:55.728273 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/77f8326b-54be-49e5-a855-647cd2cd05f8-installation-pull-secrets\") pod \"77f8326b-54be-49e5-a855-647cd2cd05f8\" (UID: \"77f8326b-54be-49e5-a855-647cd2cd05f8\") " Jul 16 16:52:55.728547 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:55.728458 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77f8326b-54be-49e5-a855-647cd2cd05f8-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "77f8326b-54be-49e5-a855-647cd2cd05f8" (UID: "77f8326b-54be-49e5-a855-647cd2cd05f8"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 16:52:55.728624 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:55.728599 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77f8326b-54be-49e5-a855-647cd2cd05f8-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "77f8326b-54be-49e5-a855-647cd2cd05f8" (UID: "77f8326b-54be-49e5-a855-647cd2cd05f8"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 16:52:55.730847 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:55.730821 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77f8326b-54be-49e5-a855-647cd2cd05f8-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "77f8326b-54be-49e5-a855-647cd2cd05f8" (UID: "77f8326b-54be-49e5-a855-647cd2cd05f8"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 16:52:55.730973 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:55.730910 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77f8326b-54be-49e5-a855-647cd2cd05f8-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "77f8326b-54be-49e5-a855-647cd2cd05f8" (UID: "77f8326b-54be-49e5-a855-647cd2cd05f8"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:52:55.731053 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:55.731020 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77f8326b-54be-49e5-a855-647cd2cd05f8-kube-api-access-hwxx4" (OuterVolumeSpecName: "kube-api-access-hwxx4") pod "77f8326b-54be-49e5-a855-647cd2cd05f8" (UID: "77f8326b-54be-49e5-a855-647cd2cd05f8"). InnerVolumeSpecName "kube-api-access-hwxx4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:52:55.731152 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:55.731068 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77f8326b-54be-49e5-a855-647cd2cd05f8-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "77f8326b-54be-49e5-a855-647cd2cd05f8" (UID: "77f8326b-54be-49e5-a855-647cd2cd05f8"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 16:52:55.731152 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:55.731073 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77f8326b-54be-49e5-a855-647cd2cd05f8-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "77f8326b-54be-49e5-a855-647cd2cd05f8" (UID: "77f8326b-54be-49e5-a855-647cd2cd05f8"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:52:55.736742 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:55.736701 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77f8326b-54be-49e5-a855-647cd2cd05f8-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "77f8326b-54be-49e5-a855-647cd2cd05f8" (UID: "77f8326b-54be-49e5-a855-647cd2cd05f8"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:52:55.829250 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:55.829176 2571 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/77f8326b-54be-49e5-a855-647cd2cd05f8-registry-certificates\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:52:55.829250 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:55.829202 2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/77f8326b-54be-49e5-a855-647cd2cd05f8-trusted-ca\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:52:55.829250 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:55.829211 2571 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/77f8326b-54be-49e5-a855-647cd2cd05f8-registry-tls\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:52:55.829250 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:55.829219 2571 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/77f8326b-54be-49e5-a855-647cd2cd05f8-ca-trust-extracted\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:52:55.829250 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:55.829226 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwxx4\" (UniqueName: \"kubernetes.io/projected/77f8326b-54be-49e5-a855-647cd2cd05f8-kube-api-access-hwxx4\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:52:55.829250 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:55.829235 2571 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/77f8326b-54be-49e5-a855-647cd2cd05f8-bound-sa-token\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:52:55.829250 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:55.829255 2571 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/77f8326b-54be-49e5-a855-647cd2cd05f8-installation-pull-secrets\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:52:55.829575 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:55.829264 2571 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/77f8326b-54be-49e5-a855-647cd2cd05f8-image-registry-private-configuration\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:52:56.383647 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:56.383613 2571 generic.go:356] "Generic (PLEG): container finished" podID="77f8326b-54be-49e5-a855-647cd2cd05f8" containerID="0b3a77772590cec77d6236db8f4f0998fcaa3615ff6ffd7d0e9634e3aef4f5be" exitCode=0 Jul 16 16:52:56.384138 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:56.383652 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" Jul 16 16:52:56.384138 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:56.383702 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" event={"ID":"77f8326b-54be-49e5-a855-647cd2cd05f8","Type":"ContainerDied","Data":"0b3a77772590cec77d6236db8f4f0998fcaa3615ff6ffd7d0e9634e3aef4f5be"} Jul 16 16:52:56.384138 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:56.383779 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5574f576bc-zfnfv" event={"ID":"77f8326b-54be-49e5-a855-647cd2cd05f8","Type":"ContainerDied","Data":"51c2e5ffe80ba0ac7c11d67519947b4e37bd9c26b565ada9f40911f2275034f5"} Jul 16 16:52:56.384138 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:56.383810 2571 scope.go:117] "RemoveContainer" containerID="0b3a77772590cec77d6236db8f4f0998fcaa3615ff6ffd7d0e9634e3aef4f5be" Jul 16 16:52:56.385594 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:56.385570 2571 generic.go:356] "Generic (PLEG): container finished" podID="37bad57e-950d-4a6a-984b-19f240be539b" containerID="e5551e447d95e9c5d78e5daccfccf9c5abda111bc0d4b02499ebc792bad47999" exitCode=0 Jul 16 16:52:56.385711 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:56.385671 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-5t9rp" event={"ID":"37bad57e-950d-4a6a-984b-19f240be539b","Type":"ContainerDied","Data":"e5551e447d95e9c5d78e5daccfccf9c5abda111bc0d4b02499ebc792bad47999"} Jul 16 16:52:56.386057 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:56.386040 2571 scope.go:117] "RemoveContainer" containerID="e5551e447d95e9c5d78e5daccfccf9c5abda111bc0d4b02499ebc792bad47999" Jul 16 16:52:56.389607 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:56.389565 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fjsn9" event={"ID":"32cd8d7c-6e88-483e-a5d7-f028311b4be3","Type":"ContainerStarted","Data":"5442c3e236f9ee6fe511d461cd8a3183c7fb6f9d38df423fd0e5c2ec595a72d8"} Jul 16 16:52:56.397353 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:56.397335 2571 scope.go:117] "RemoveContainer" containerID="0b3a77772590cec77d6236db8f4f0998fcaa3615ff6ffd7d0e9634e3aef4f5be" Jul 16 16:52:56.397609 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:52:56.397582 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b3a77772590cec77d6236db8f4f0998fcaa3615ff6ffd7d0e9634e3aef4f5be\": container with ID starting with 0b3a77772590cec77d6236db8f4f0998fcaa3615ff6ffd7d0e9634e3aef4f5be not found: ID does not exist" containerID="0b3a77772590cec77d6236db8f4f0998fcaa3615ff6ffd7d0e9634e3aef4f5be" Jul 16 16:52:56.397672 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:56.397623 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b3a77772590cec77d6236db8f4f0998fcaa3615ff6ffd7d0e9634e3aef4f5be"} err="failed to get container status \"0b3a77772590cec77d6236db8f4f0998fcaa3615ff6ffd7d0e9634e3aef4f5be\": rpc error: code = NotFound desc = could not find container \"0b3a77772590cec77d6236db8f4f0998fcaa3615ff6ffd7d0e9634e3aef4f5be\": container with ID starting with 0b3a77772590cec77d6236db8f4f0998fcaa3615ff6ffd7d0e9634e3aef4f5be not found: ID does not exist" Jul 16 16:52:56.448507 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:56.448485 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5574f576bc-zfnfv"] Jul 16 16:52:56.452258 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:56.452236 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5574f576bc-zfnfv"] Jul 16 16:52:56.776243 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:56.776201 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77f8326b-54be-49e5-a855-647cd2cd05f8" path="/var/lib/kubelet/pods/77f8326b-54be-49e5-a855-647cd2cd05f8/volumes" Jul 16 16:52:57.395878 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:57.395841 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-5t9rp" event={"ID":"37bad57e-950d-4a6a-984b-19f240be539b","Type":"ContainerStarted","Data":"d595476dee4573ba6b09f331b46b35e8714aa13373a142f0cfc16b2909b6d000"} Jul 16 16:52:58.554782 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:58.554722 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2341bdcc-01b6-426c-8189-71be7fdc2e8b-metrics-certs\") pod \"network-metrics-daemon-4p6nc\" (UID: \"2341bdcc-01b6-426c-8189-71be7fdc2e8b\") " pod="openshift-multus/network-metrics-daemon-4p6nc" Jul 16 16:52:58.557014 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:58.556986 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2341bdcc-01b6-426c-8189-71be7fdc2e8b-metrics-certs\") pod \"network-metrics-daemon-4p6nc\" (UID: \"2341bdcc-01b6-426c-8189-71be7fdc2e8b\") " pod="openshift-multus/network-metrics-daemon-4p6nc" Jul 16 16:52:58.600289 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:58.600256 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-s2b72\"" Jul 16 16:52:58.607163 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:58.607139 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4p6nc" Jul 16 16:52:58.819909 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:58.819838 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4p6nc"] Jul 16 16:52:58.825106 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:52:58.825077 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2341bdcc_01b6_426c_8189_71be7fdc2e8b.slice/crio-354a7a039781e764ae906c417085462304107cc0ac3895285bbe6ae0aba2d858 WatchSource:0}: Error finding container 354a7a039781e764ae906c417085462304107cc0ac3895285bbe6ae0aba2d858: Status 404 returned error can't find the container with id 354a7a039781e764ae906c417085462304107cc0ac3895285bbe6ae0aba2d858 Jul 16 16:52:59.402994 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:52:59.402958 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4p6nc" event={"ID":"2341bdcc-01b6-426c-8189-71be7fdc2e8b","Type":"ContainerStarted","Data":"354a7a039781e764ae906c417085462304107cc0ac3895285bbe6ae0aba2d858"} Jul 16 16:53:01.412638 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:53:01.412598 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4p6nc" event={"ID":"2341bdcc-01b6-426c-8189-71be7fdc2e8b","Type":"ContainerStarted","Data":"d30e180cb443ac96c22b4d344d5abe0461354003868b4ca9400f72affc0702df"} Jul 16 16:53:01.413078 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:53:01.412656 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4p6nc" event={"ID":"2341bdcc-01b6-426c-8189-71be7fdc2e8b","Type":"ContainerStarted","Data":"4629143f3fc8c2148bc4ab759ebd1b94641671564fd38b5d2e184c636b6db257"} Jul 16 16:53:01.431570 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:53:01.431504 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-4p6nc" podStartSLOduration=131.594493233 podStartE2EDuration="2m13.431490607s" totalImagesPullingTime="1.836997374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:50:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:53:01.429967459 +0000 UTC m=+133.171542754" watchObservedRunningTime="2026-07-16 16:53:01.431490607 +0000 UTC m=+133.173065901" Jul 16 16:53:11.453945 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:53:11.453912 2571 generic.go:356] "Generic (PLEG): container finished" podID="3d9162b4-f5a7-4aff-afd6-73383ad17f4a" containerID="f1fc337f18983f145253f656eaa342f2d9de39d9d238af7b623bf2a5a6ddb67c" exitCode=0 Jul 16 16:53:11.454333 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:53:11.453964 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-6zvr8" event={"ID":"3d9162b4-f5a7-4aff-afd6-73383ad17f4a","Type":"ContainerDied","Data":"f1fc337f18983f145253f656eaa342f2d9de39d9d238af7b623bf2a5a6ddb67c"} Jul 16 16:53:11.454333 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:53:11.454238 2571 scope.go:117] "RemoveContainer" containerID="f1fc337f18983f145253f656eaa342f2d9de39d9d238af7b623bf2a5a6ddb67c" Jul 16 16:53:12.459170 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:53:12.459131 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-6zvr8" event={"ID":"3d9162b4-f5a7-4aff-afd6-73383ad17f4a","Type":"ContainerStarted","Data":"75f2e5df733cab848833bb47e54c0d930b7ab28d7dd08bbc9f27207fbfe46f9d"} Jul 16 16:53:56.631406 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:53:56.631377 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-55f47d678-psdcp"] Jul 16 16:53:56.631840 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:53:56.631627 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77f8326b-54be-49e5-a855-647cd2cd05f8" containerName="registry" Jul 16 16:53:56.631840 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:53:56.631638 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="77f8326b-54be-49e5-a855-647cd2cd05f8" containerName="registry" Jul 16 16:53:56.631840 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:53:56.631693 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="77f8326b-54be-49e5-a855-647cd2cd05f8" containerName="registry" Jul 16 16:53:56.635950 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:53:56.635927 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55f47d678-psdcp" Jul 16 16:53:56.648327 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:53:56.648303 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55f47d678-psdcp"] Jul 16 16:53:56.695596 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:53:56.695569 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b0833a28-5c2e-4656-877e-ba884c88cf4d-service-ca\") pod \"console-55f47d678-psdcp\" (UID: \"b0833a28-5c2e-4656-877e-ba884c88cf4d\") " pod="openshift-console/console-55f47d678-psdcp" Jul 16 16:53:56.695776 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:53:56.695607 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0833a28-5c2e-4656-877e-ba884c88cf4d-trusted-ca-bundle\") pod \"console-55f47d678-psdcp\" (UID: \"b0833a28-5c2e-4656-877e-ba884c88cf4d\") " pod="openshift-console/console-55f47d678-psdcp" Jul 16 16:53:56.695776 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:53:56.695625 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b0833a28-5c2e-4656-877e-ba884c88cf4d-console-serving-cert\") pod \"console-55f47d678-psdcp\" (UID: \"b0833a28-5c2e-4656-877e-ba884c88cf4d\") " pod="openshift-console/console-55f47d678-psdcp" Jul 16 16:53:56.695776 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:53:56.695714 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vskdd\" (UniqueName: \"kubernetes.io/projected/b0833a28-5c2e-4656-877e-ba884c88cf4d-kube-api-access-vskdd\") pod \"console-55f47d678-psdcp\" (UID: \"b0833a28-5c2e-4656-877e-ba884c88cf4d\") " pod="openshift-console/console-55f47d678-psdcp" Jul 16 16:53:56.695776 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:53:56.695762 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b0833a28-5c2e-4656-877e-ba884c88cf4d-console-config\") pod \"console-55f47d678-psdcp\" (UID: \"b0833a28-5c2e-4656-877e-ba884c88cf4d\") " pod="openshift-console/console-55f47d678-psdcp" Jul 16 16:53:56.695942 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:53:56.695782 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b0833a28-5c2e-4656-877e-ba884c88cf4d-oauth-serving-cert\") pod \"console-55f47d678-psdcp\" (UID: \"b0833a28-5c2e-4656-877e-ba884c88cf4d\") " pod="openshift-console/console-55f47d678-psdcp" Jul 16 16:53:56.695942 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:53:56.695800 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b0833a28-5c2e-4656-877e-ba884c88cf4d-console-oauth-config\") pod \"console-55f47d678-psdcp\" (UID: \"b0833a28-5c2e-4656-877e-ba884c88cf4d\") " pod="openshift-console/console-55f47d678-psdcp" Jul 16 16:53:56.796682 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:53:56.796650 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b0833a28-5c2e-4656-877e-ba884c88cf4d-service-ca\") pod \"console-55f47d678-psdcp\" (UID: \"b0833a28-5c2e-4656-877e-ba884c88cf4d\") " pod="openshift-console/console-55f47d678-psdcp" Jul 16 16:53:56.796682 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:53:56.796686 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0833a28-5c2e-4656-877e-ba884c88cf4d-trusted-ca-bundle\") pod \"console-55f47d678-psdcp\" (UID: \"b0833a28-5c2e-4656-877e-ba884c88cf4d\") " pod="openshift-console/console-55f47d678-psdcp" Jul 16 16:53:56.796963 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:53:56.796705 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b0833a28-5c2e-4656-877e-ba884c88cf4d-console-serving-cert\") pod \"console-55f47d678-psdcp\" (UID: \"b0833a28-5c2e-4656-877e-ba884c88cf4d\") " pod="openshift-console/console-55f47d678-psdcp" Jul 16 16:53:56.796963 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:53:56.796749 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vskdd\" (UniqueName: \"kubernetes.io/projected/b0833a28-5c2e-4656-877e-ba884c88cf4d-kube-api-access-vskdd\") pod \"console-55f47d678-psdcp\" (UID: \"b0833a28-5c2e-4656-877e-ba884c88cf4d\") " pod="openshift-console/console-55f47d678-psdcp" Jul 16 16:53:56.796963 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:53:56.796874 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b0833a28-5c2e-4656-877e-ba884c88cf4d-console-config\") pod \"console-55f47d678-psdcp\" (UID: \"b0833a28-5c2e-4656-877e-ba884c88cf4d\") " pod="openshift-console/console-55f47d678-psdcp" Jul 16 16:53:56.796963 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:53:56.796906 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b0833a28-5c2e-4656-877e-ba884c88cf4d-oauth-serving-cert\") pod \"console-55f47d678-psdcp\" (UID: \"b0833a28-5c2e-4656-877e-ba884c88cf4d\") " pod="openshift-console/console-55f47d678-psdcp" Jul 16 16:53:56.796963 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:53:56.796936 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b0833a28-5c2e-4656-877e-ba884c88cf4d-console-oauth-config\") pod \"console-55f47d678-psdcp\" (UID: \"b0833a28-5c2e-4656-877e-ba884c88cf4d\") " pod="openshift-console/console-55f47d678-psdcp" Jul 16 16:53:56.797489 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:53:56.797453 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b0833a28-5c2e-4656-877e-ba884c88cf4d-service-ca\") pod \"console-55f47d678-psdcp\" (UID: \"b0833a28-5c2e-4656-877e-ba884c88cf4d\") " pod="openshift-console/console-55f47d678-psdcp" Jul 16 16:53:56.797614 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:53:56.797576 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b0833a28-5c2e-4656-877e-ba884c88cf4d-console-config\") pod \"console-55f47d678-psdcp\" (UID: \"b0833a28-5c2e-4656-877e-ba884c88cf4d\") " pod="openshift-console/console-55f47d678-psdcp" Jul 16 16:53:56.797669 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:53:56.797610 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b0833a28-5c2e-4656-877e-ba884c88cf4d-oauth-serving-cert\") pod \"console-55f47d678-psdcp\" (UID: \"b0833a28-5c2e-4656-877e-ba884c88cf4d\") " pod="openshift-console/console-55f47d678-psdcp" Jul 16 16:53:56.797669 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:53:56.797611 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0833a28-5c2e-4656-877e-ba884c88cf4d-trusted-ca-bundle\") pod \"console-55f47d678-psdcp\" (UID: \"b0833a28-5c2e-4656-877e-ba884c88cf4d\") " pod="openshift-console/console-55f47d678-psdcp" Jul 16 16:53:56.799259 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:53:56.799233 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b0833a28-5c2e-4656-877e-ba884c88cf4d-console-serving-cert\") pod \"console-55f47d678-psdcp\" (UID: \"b0833a28-5c2e-4656-877e-ba884c88cf4d\") " pod="openshift-console/console-55f47d678-psdcp" Jul 16 16:53:56.799334 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:53:56.799276 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b0833a28-5c2e-4656-877e-ba884c88cf4d-console-oauth-config\") pod \"console-55f47d678-psdcp\" (UID: \"b0833a28-5c2e-4656-877e-ba884c88cf4d\") " pod="openshift-console/console-55f47d678-psdcp" Jul 16 16:53:56.806906 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:53:56.806884 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vskdd\" (UniqueName: \"kubernetes.io/projected/b0833a28-5c2e-4656-877e-ba884c88cf4d-kube-api-access-vskdd\") pod \"console-55f47d678-psdcp\" (UID: \"b0833a28-5c2e-4656-877e-ba884c88cf4d\") " pod="openshift-console/console-55f47d678-psdcp" Jul 16 16:53:56.945393 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:53:56.945366 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55f47d678-psdcp" Jul 16 16:53:57.092132 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:53:57.092104 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55f47d678-psdcp"] Jul 16 16:53:57.094603 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:53:57.094571 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0833a28_5c2e_4656_877e_ba884c88cf4d.slice/crio-63408288c1a89ee44d9809cda53295ec114fe4725b23305b8f36038f552a9947 WatchSource:0}: Error finding container 63408288c1a89ee44d9809cda53295ec114fe4725b23305b8f36038f552a9947: Status 404 returned error can't find the container with id 63408288c1a89ee44d9809cda53295ec114fe4725b23305b8f36038f552a9947 Jul 16 16:53:57.624252 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:53:57.624214 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55f47d678-psdcp" event={"ID":"b0833a28-5c2e-4656-877e-ba884c88cf4d","Type":"ContainerStarted","Data":"477f57a1432568a0e0f4d0a64fcdd1817b9190f3e5f20ae759fe63f6f0874905"} Jul 16 16:53:57.624415 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:53:57.624265 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55f47d678-psdcp" event={"ID":"b0833a28-5c2e-4656-877e-ba884c88cf4d","Type":"ContainerStarted","Data":"63408288c1a89ee44d9809cda53295ec114fe4725b23305b8f36038f552a9947"} Jul 16 16:53:57.645603 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:53:57.645551 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-55f47d678-psdcp" podStartSLOduration=1.6455377150000001 podStartE2EDuration="1.645537715s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:53:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:53:57.644848471 +0000 UTC m=+189.386423763" watchObservedRunningTime="2026-07-16 16:53:57.645537715 +0000 UTC m=+189.387113012" Jul 16 16:54:06.945885 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:54:06.945855 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-55f47d678-psdcp" Jul 16 16:54:06.946330 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:54:06.945904 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-55f47d678-psdcp" Jul 16 16:54:06.950929 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:54:06.950902 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-55f47d678-psdcp" Jul 16 16:54:07.665777 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:54:07.665741 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-55f47d678-psdcp" Jul 16 16:54:07.719413 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:54:07.719378 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c758bf489-xhhwx"] Jul 16 16:54:32.743140 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:54:32.743083 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-c758bf489-xhhwx" podUID="1edbc8b0-0e10-4256-af6f-b8556c599efa" containerName="console" containerID="cri-o://80311e91b90b3919ffa6672bdc544ff0dac85b37e0f7dbe97ce9ddfca0e073fa" gracePeriod=15 Jul 16 16:54:32.990315 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:54:32.990290 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c758bf489-xhhwx" Jul 16 16:54:33.080833 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:54:33.080743 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1edbc8b0-0e10-4256-af6f-b8556c599efa-console-oauth-config\") pod \"1edbc8b0-0e10-4256-af6f-b8556c599efa\" (UID: \"1edbc8b0-0e10-4256-af6f-b8556c599efa\") " Jul 16 16:54:33.080833 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:54:33.080792 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j8gjs\" (UniqueName: \"kubernetes.io/projected/1edbc8b0-0e10-4256-af6f-b8556c599efa-kube-api-access-j8gjs\") pod \"1edbc8b0-0e10-4256-af6f-b8556c599efa\" (UID: \"1edbc8b0-0e10-4256-af6f-b8556c599efa\") " Jul 16 16:54:33.080833 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:54:33.080827 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1edbc8b0-0e10-4256-af6f-b8556c599efa-console-serving-cert\") pod \"1edbc8b0-0e10-4256-af6f-b8556c599efa\" (UID: \"1edbc8b0-0e10-4256-af6f-b8556c599efa\") " Jul 16 16:54:33.081104 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:54:33.080847 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1edbc8b0-0e10-4256-af6f-b8556c599efa-oauth-serving-cert\") pod \"1edbc8b0-0e10-4256-af6f-b8556c599efa\" (UID: \"1edbc8b0-0e10-4256-af6f-b8556c599efa\") " Jul 16 16:54:33.081104 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:54:33.080880 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1edbc8b0-0e10-4256-af6f-b8556c599efa-console-config\") pod \"1edbc8b0-0e10-4256-af6f-b8556c599efa\" (UID: \"1edbc8b0-0e10-4256-af6f-b8556c599efa\") " Jul 16 16:54:33.081104 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:54:33.080920 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1edbc8b0-0e10-4256-af6f-b8556c599efa-service-ca\") pod \"1edbc8b0-0e10-4256-af6f-b8556c599efa\" (UID: \"1edbc8b0-0e10-4256-af6f-b8556c599efa\") " Jul 16 16:54:33.081104 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:54:33.080947 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1edbc8b0-0e10-4256-af6f-b8556c599efa-trusted-ca-bundle\") pod \"1edbc8b0-0e10-4256-af6f-b8556c599efa\" (UID: \"1edbc8b0-0e10-4256-af6f-b8556c599efa\") " Jul 16 16:54:33.081386 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:54:33.081347 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1edbc8b0-0e10-4256-af6f-b8556c599efa-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1edbc8b0-0e10-4256-af6f-b8556c599efa" (UID: "1edbc8b0-0e10-4256-af6f-b8556c599efa"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 16:54:33.081386 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:54:33.081365 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1edbc8b0-0e10-4256-af6f-b8556c599efa-console-config" (OuterVolumeSpecName: "console-config") pod "1edbc8b0-0e10-4256-af6f-b8556c599efa" (UID: "1edbc8b0-0e10-4256-af6f-b8556c599efa"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 16:54:33.081516 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:54:33.081471 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1edbc8b0-0e10-4256-af6f-b8556c599efa-service-ca" (OuterVolumeSpecName: "service-ca") pod "1edbc8b0-0e10-4256-af6f-b8556c599efa" (UID: "1edbc8b0-0e10-4256-af6f-b8556c599efa"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 16:54:33.081605 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:54:33.081581 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1edbc8b0-0e10-4256-af6f-b8556c599efa-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1edbc8b0-0e10-4256-af6f-b8556c599efa" (UID: "1edbc8b0-0e10-4256-af6f-b8556c599efa"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 16:54:33.083172 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:54:33.083151 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1edbc8b0-0e10-4256-af6f-b8556c599efa-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1edbc8b0-0e10-4256-af6f-b8556c599efa" (UID: "1edbc8b0-0e10-4256-af6f-b8556c599efa"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 16:54:33.083250 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:54:33.083173 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1edbc8b0-0e10-4256-af6f-b8556c599efa-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1edbc8b0-0e10-4256-af6f-b8556c599efa" (UID: "1edbc8b0-0e10-4256-af6f-b8556c599efa"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 16:54:33.083250 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:54:33.083195 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1edbc8b0-0e10-4256-af6f-b8556c599efa-kube-api-access-j8gjs" (OuterVolumeSpecName: "kube-api-access-j8gjs") pod "1edbc8b0-0e10-4256-af6f-b8556c599efa" (UID: "1edbc8b0-0e10-4256-af6f-b8556c599efa"). InnerVolumeSpecName "kube-api-access-j8gjs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:54:33.182253 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:54:33.182215 2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1edbc8b0-0e10-4256-af6f-b8556c599efa-console-serving-cert\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:54:33.182253 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:54:33.182246 2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1edbc8b0-0e10-4256-af6f-b8556c599efa-oauth-serving-cert\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:54:33.182253 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:54:33.182259 2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1edbc8b0-0e10-4256-af6f-b8556c599efa-console-config\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:54:33.182477 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:54:33.182271 2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1edbc8b0-0e10-4256-af6f-b8556c599efa-service-ca\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:54:33.182477 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:54:33.182281 2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1edbc8b0-0e10-4256-af6f-b8556c599efa-trusted-ca-bundle\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:54:33.182477 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:54:33.182292 2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1edbc8b0-0e10-4256-af6f-b8556c599efa-console-oauth-config\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:54:33.182477 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:54:33.182303 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j8gjs\" (UniqueName: \"kubernetes.io/projected/1edbc8b0-0e10-4256-af6f-b8556c599efa-kube-api-access-j8gjs\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:54:33.757256 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:54:33.757218 2571 generic.go:356] "Generic (PLEG): container finished" podID="1edbc8b0-0e10-4256-af6f-b8556c599efa" containerID="80311e91b90b3919ffa6672bdc544ff0dac85b37e0f7dbe97ce9ddfca0e073fa" exitCode=0 Jul 16 16:54:33.757720 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:54:33.757282 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c758bf489-xhhwx" Jul 16 16:54:33.757720 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:54:33.757290 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c758bf489-xhhwx" event={"ID":"1edbc8b0-0e10-4256-af6f-b8556c599efa","Type":"ContainerDied","Data":"80311e91b90b3919ffa6672bdc544ff0dac85b37e0f7dbe97ce9ddfca0e073fa"} Jul 16 16:54:33.757720 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:54:33.757358 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c758bf489-xhhwx" event={"ID":"1edbc8b0-0e10-4256-af6f-b8556c599efa","Type":"ContainerDied","Data":"897441209c4201ce1a50804b8e8dd154fd5debd55676d8656ea1d8803e833987"} Jul 16 16:54:33.757720 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:54:33.757380 2571 scope.go:117] "RemoveContainer" containerID="80311e91b90b3919ffa6672bdc544ff0dac85b37e0f7dbe97ce9ddfca0e073fa" Jul 16 16:54:33.765468 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:54:33.765443 2571 scope.go:117] "RemoveContainer" containerID="80311e91b90b3919ffa6672bdc544ff0dac85b37e0f7dbe97ce9ddfca0e073fa" Jul 16 16:54:33.765783 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:54:33.765766 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80311e91b90b3919ffa6672bdc544ff0dac85b37e0f7dbe97ce9ddfca0e073fa\": container with ID starting with 80311e91b90b3919ffa6672bdc544ff0dac85b37e0f7dbe97ce9ddfca0e073fa not found: ID does not exist" containerID="80311e91b90b3919ffa6672bdc544ff0dac85b37e0f7dbe97ce9ddfca0e073fa" Jul 16 16:54:33.765852 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:54:33.765791 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80311e91b90b3919ffa6672bdc544ff0dac85b37e0f7dbe97ce9ddfca0e073fa"} err="failed to get container status \"80311e91b90b3919ffa6672bdc544ff0dac85b37e0f7dbe97ce9ddfca0e073fa\": rpc error: code = NotFound desc = could not find container \"80311e91b90b3919ffa6672bdc544ff0dac85b37e0f7dbe97ce9ddfca0e073fa\": container with ID starting with 80311e91b90b3919ffa6672bdc544ff0dac85b37e0f7dbe97ce9ddfca0e073fa not found: ID does not exist" Jul 16 16:54:33.780229 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:54:33.780203 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c758bf489-xhhwx"] Jul 16 16:54:33.787398 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:54:33.787376 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-c758bf489-xhhwx"] Jul 16 16:54:34.775955 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:54:34.775922 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1edbc8b0-0e10-4256-af6f-b8556c599efa" path="/var/lib/kubelet/pods/1edbc8b0-0e10-4256-af6f-b8556c599efa/volumes" Jul 16 16:55:05.712476 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:05.712437 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-9dd7fc48d-gj566"] Jul 16 16:55:05.713119 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:05.712966 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1edbc8b0-0e10-4256-af6f-b8556c599efa" containerName="console" Jul 16 16:55:05.713119 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:05.712988 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1edbc8b0-0e10-4256-af6f-b8556c599efa" containerName="console" Jul 16 16:55:05.713255 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:05.713138 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="1edbc8b0-0e10-4256-af6f-b8556c599efa" containerName="console" Jul 16 16:55:05.717559 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:05.717531 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9dd7fc48d-gj566" Jul 16 16:55:05.728958 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:05.728932 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9dd7fc48d-gj566"] Jul 16 16:55:05.826176 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:05.826144 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-console-config\") pod \"console-9dd7fc48d-gj566\" (UID: \"6e52ccea-4b86-433c-b6aa-cbe0fa0870a6\") " pod="openshift-console/console-9dd7fc48d-gj566" Jul 16 16:55:05.826390 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:05.826203 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-console-oauth-config\") pod \"console-9dd7fc48d-gj566\" (UID: \"6e52ccea-4b86-433c-b6aa-cbe0fa0870a6\") " pod="openshift-console/console-9dd7fc48d-gj566" Jul 16 16:55:05.826390 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:05.826280 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-service-ca\") pod \"console-9dd7fc48d-gj566\" (UID: \"6e52ccea-4b86-433c-b6aa-cbe0fa0870a6\") " pod="openshift-console/console-9dd7fc48d-gj566" Jul 16 16:55:05.826390 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:05.826345 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-console-serving-cert\") pod \"console-9dd7fc48d-gj566\" (UID: \"6e52ccea-4b86-433c-b6aa-cbe0fa0870a6\") " pod="openshift-console/console-9dd7fc48d-gj566" Jul 16 16:55:05.826390 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:05.826380 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-oauth-serving-cert\") pod \"console-9dd7fc48d-gj566\" (UID: \"6e52ccea-4b86-433c-b6aa-cbe0fa0870a6\") " pod="openshift-console/console-9dd7fc48d-gj566" Jul 16 16:55:05.826579 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:05.826429 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qttxc\" (UniqueName: \"kubernetes.io/projected/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-kube-api-access-qttxc\") pod \"console-9dd7fc48d-gj566\" (UID: \"6e52ccea-4b86-433c-b6aa-cbe0fa0870a6\") " pod="openshift-console/console-9dd7fc48d-gj566" Jul 16 16:55:05.826579 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:05.826457 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-trusted-ca-bundle\") pod \"console-9dd7fc48d-gj566\" (UID: \"6e52ccea-4b86-433c-b6aa-cbe0fa0870a6\") " pod="openshift-console/console-9dd7fc48d-gj566" Jul 16 16:55:05.927675 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:05.927636 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qttxc\" (UniqueName: \"kubernetes.io/projected/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-kube-api-access-qttxc\") pod \"console-9dd7fc48d-gj566\" (UID: \"6e52ccea-4b86-433c-b6aa-cbe0fa0870a6\") " pod="openshift-console/console-9dd7fc48d-gj566" Jul 16 16:55:05.927675 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:05.927676 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-trusted-ca-bundle\") pod \"console-9dd7fc48d-gj566\" (UID: \"6e52ccea-4b86-433c-b6aa-cbe0fa0870a6\") " pod="openshift-console/console-9dd7fc48d-gj566" Jul 16 16:55:05.927972 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:05.927704 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-console-config\") pod \"console-9dd7fc48d-gj566\" (UID: \"6e52ccea-4b86-433c-b6aa-cbe0fa0870a6\") " pod="openshift-console/console-9dd7fc48d-gj566" Jul 16 16:55:05.927972 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:05.927763 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-console-oauth-config\") pod \"console-9dd7fc48d-gj566\" (UID: \"6e52ccea-4b86-433c-b6aa-cbe0fa0870a6\") " pod="openshift-console/console-9dd7fc48d-gj566" Jul 16 16:55:05.927972 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:05.927938 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-service-ca\") pod \"console-9dd7fc48d-gj566\" (UID: \"6e52ccea-4b86-433c-b6aa-cbe0fa0870a6\") " pod="openshift-console/console-9dd7fc48d-gj566" Jul 16 16:55:05.928129 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:05.927992 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-console-serving-cert\") pod \"console-9dd7fc48d-gj566\" (UID: \"6e52ccea-4b86-433c-b6aa-cbe0fa0870a6\") " pod="openshift-console/console-9dd7fc48d-gj566" Jul 16 16:55:05.928129 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:05.928017 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-oauth-serving-cert\") pod \"console-9dd7fc48d-gj566\" (UID: \"6e52ccea-4b86-433c-b6aa-cbe0fa0870a6\") " pod="openshift-console/console-9dd7fc48d-gj566" Jul 16 16:55:05.928467 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:05.928433 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-console-config\") pod \"console-9dd7fc48d-gj566\" (UID: \"6e52ccea-4b86-433c-b6aa-cbe0fa0870a6\") " pod="openshift-console/console-9dd7fc48d-gj566" Jul 16 16:55:05.928610 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:05.928592 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-trusted-ca-bundle\") pod \"console-9dd7fc48d-gj566\" (UID: \"6e52ccea-4b86-433c-b6aa-cbe0fa0870a6\") " pod="openshift-console/console-9dd7fc48d-gj566" Jul 16 16:55:05.928673 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:05.928590 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-service-ca\") pod \"console-9dd7fc48d-gj566\" (UID: \"6e52ccea-4b86-433c-b6aa-cbe0fa0870a6\") " pod="openshift-console/console-9dd7fc48d-gj566" Jul 16 16:55:05.929145 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:05.929125 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-oauth-serving-cert\") pod \"console-9dd7fc48d-gj566\" (UID: \"6e52ccea-4b86-433c-b6aa-cbe0fa0870a6\") " pod="openshift-console/console-9dd7fc48d-gj566" Jul 16 16:55:05.930499 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:05.930473 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-console-serving-cert\") pod \"console-9dd7fc48d-gj566\" (UID: \"6e52ccea-4b86-433c-b6aa-cbe0fa0870a6\") " pod="openshift-console/console-9dd7fc48d-gj566" Jul 16 16:55:05.930589 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:05.930519 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-console-oauth-config\") pod \"console-9dd7fc48d-gj566\" (UID: \"6e52ccea-4b86-433c-b6aa-cbe0fa0870a6\") " pod="openshift-console/console-9dd7fc48d-gj566" Jul 16 16:55:05.937842 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:05.937823 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qttxc\" (UniqueName: \"kubernetes.io/projected/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-kube-api-access-qttxc\") pod \"console-9dd7fc48d-gj566\" (UID: \"6e52ccea-4b86-433c-b6aa-cbe0fa0870a6\") " pod="openshift-console/console-9dd7fc48d-gj566" Jul 16 16:55:06.029190 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:06.029087 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9dd7fc48d-gj566" Jul 16 16:55:06.164456 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:06.164420 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9dd7fc48d-gj566"] Jul 16 16:55:06.168378 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:55:06.168351 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e52ccea_4b86_433c_b6aa_cbe0fa0870a6.slice/crio-da0d451b671f22a465302392c85aee0bc1166ede40ceab2681635a44125c9dae WatchSource:0}: Error finding container da0d451b671f22a465302392c85aee0bc1166ede40ceab2681635a44125c9dae: Status 404 returned error can't find the container with id da0d451b671f22a465302392c85aee0bc1166ede40ceab2681635a44125c9dae Jul 16 16:55:06.876614 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:06.876576 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9dd7fc48d-gj566" event={"ID":"6e52ccea-4b86-433c-b6aa-cbe0fa0870a6","Type":"ContainerStarted","Data":"4b7582a111360b38f99bea72882ce5cee587c6667118c31a6275015aad58c477"} Jul 16 16:55:06.877141 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:06.876633 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9dd7fc48d-gj566" event={"ID":"6e52ccea-4b86-433c-b6aa-cbe0fa0870a6","Type":"ContainerStarted","Data":"da0d451b671f22a465302392c85aee0bc1166ede40ceab2681635a44125c9dae"} Jul 16 16:55:06.906106 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:06.906056 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-9dd7fc48d-gj566" podStartSLOduration=1.9060400899999999 podStartE2EDuration="1.90604009s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:55:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:55:06.905060297 +0000 UTC m=+258.646635597" watchObservedRunningTime="2026-07-16 16:55:06.90604009 +0000 UTC m=+258.647615387" Jul 16 16:55:16.030000 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:16.029915 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-9dd7fc48d-gj566" Jul 16 16:55:16.030545 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:16.030237 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-9dd7fc48d-gj566" Jul 16 16:55:16.034454 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:16.034433 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-9dd7fc48d-gj566" Jul 16 16:55:16.916744 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:16.916699 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-9dd7fc48d-gj566" Jul 16 16:55:16.974088 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:16.974055 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55f47d678-psdcp"] Jul 16 16:55:41.995361 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:41.995320 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-55f47d678-psdcp" podUID="b0833a28-5c2e-4656-877e-ba884c88cf4d" containerName="console" containerID="cri-o://477f57a1432568a0e0f4d0a64fcdd1817b9190f3e5f20ae759fe63f6f0874905" gracePeriod=15 Jul 16 16:55:42.231921 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:42.231896 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55f47d678-psdcp" Jul 16 16:55:42.302443 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:42.302357 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b0833a28-5c2e-4656-877e-ba884c88cf4d-console-oauth-config\") pod \"b0833a28-5c2e-4656-877e-ba884c88cf4d\" (UID: \"b0833a28-5c2e-4656-877e-ba884c88cf4d\") " Jul 16 16:55:42.302443 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:42.302403 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b0833a28-5c2e-4656-877e-ba884c88cf4d-console-config\") pod \"b0833a28-5c2e-4656-877e-ba884c88cf4d\" (UID: \"b0833a28-5c2e-4656-877e-ba884c88cf4d\") " Jul 16 16:55:42.302443 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:42.302436 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vskdd\" (UniqueName: \"kubernetes.io/projected/b0833a28-5c2e-4656-877e-ba884c88cf4d-kube-api-access-vskdd\") pod \"b0833a28-5c2e-4656-877e-ba884c88cf4d\" (UID: \"b0833a28-5c2e-4656-877e-ba884c88cf4d\") " Jul 16 16:55:42.302710 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:42.302538 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b0833a28-5c2e-4656-877e-ba884c88cf4d-oauth-serving-cert\") pod \"b0833a28-5c2e-4656-877e-ba884c88cf4d\" (UID: \"b0833a28-5c2e-4656-877e-ba884c88cf4d\") " Jul 16 16:55:42.302710 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:42.302572 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b0833a28-5c2e-4656-877e-ba884c88cf4d-service-ca\") pod \"b0833a28-5c2e-4656-877e-ba884c88cf4d\" (UID: \"b0833a28-5c2e-4656-877e-ba884c88cf4d\") " Jul 16 16:55:42.302710 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:42.302613 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0833a28-5c2e-4656-877e-ba884c88cf4d-trusted-ca-bundle\") pod \"b0833a28-5c2e-4656-877e-ba884c88cf4d\" (UID: \"b0833a28-5c2e-4656-877e-ba884c88cf4d\") " Jul 16 16:55:42.302710 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:42.302660 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b0833a28-5c2e-4656-877e-ba884c88cf4d-console-serving-cert\") pod \"b0833a28-5c2e-4656-877e-ba884c88cf4d\" (UID: \"b0833a28-5c2e-4656-877e-ba884c88cf4d\") " Jul 16 16:55:42.302952 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:42.302917 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0833a28-5c2e-4656-877e-ba884c88cf4d-console-config" (OuterVolumeSpecName: "console-config") pod "b0833a28-5c2e-4656-877e-ba884c88cf4d" (UID: "b0833a28-5c2e-4656-877e-ba884c88cf4d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 16:55:42.303018 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:42.302993 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0833a28-5c2e-4656-877e-ba884c88cf4d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b0833a28-5c2e-4656-877e-ba884c88cf4d" (UID: "b0833a28-5c2e-4656-877e-ba884c88cf4d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 16:55:42.303136 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:42.303101 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0833a28-5c2e-4656-877e-ba884c88cf4d-service-ca" (OuterVolumeSpecName: "service-ca") pod "b0833a28-5c2e-4656-877e-ba884c88cf4d" (UID: "b0833a28-5c2e-4656-877e-ba884c88cf4d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 16:55:42.303251 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:42.303144 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0833a28-5c2e-4656-877e-ba884c88cf4d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b0833a28-5c2e-4656-877e-ba884c88cf4d" (UID: "b0833a28-5c2e-4656-877e-ba884c88cf4d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 16:55:42.304564 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:42.304541 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0833a28-5c2e-4656-877e-ba884c88cf4d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b0833a28-5c2e-4656-877e-ba884c88cf4d" (UID: "b0833a28-5c2e-4656-877e-ba884c88cf4d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 16:55:42.304620 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:42.304572 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0833a28-5c2e-4656-877e-ba884c88cf4d-kube-api-access-vskdd" (OuterVolumeSpecName: "kube-api-access-vskdd") pod "b0833a28-5c2e-4656-877e-ba884c88cf4d" (UID: "b0833a28-5c2e-4656-877e-ba884c88cf4d"). InnerVolumeSpecName "kube-api-access-vskdd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:55:42.304686 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:42.304672 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0833a28-5c2e-4656-877e-ba884c88cf4d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b0833a28-5c2e-4656-877e-ba884c88cf4d" (UID: "b0833a28-5c2e-4656-877e-ba884c88cf4d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 16:55:42.403432 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:42.403392 2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b0833a28-5c2e-4656-877e-ba884c88cf4d-console-oauth-config\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:55:42.403432 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:42.403428 2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b0833a28-5c2e-4656-877e-ba884c88cf4d-console-config\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:55:42.403432 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:42.403438 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vskdd\" (UniqueName: \"kubernetes.io/projected/b0833a28-5c2e-4656-877e-ba884c88cf4d-kube-api-access-vskdd\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:55:42.403656 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:42.403447 2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b0833a28-5c2e-4656-877e-ba884c88cf4d-oauth-serving-cert\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:55:42.403656 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:42.403456 2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b0833a28-5c2e-4656-877e-ba884c88cf4d-service-ca\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:55:42.403656 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:42.403463 2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0833a28-5c2e-4656-877e-ba884c88cf4d-trusted-ca-bundle\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:55:42.403656 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:42.403471 2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b0833a28-5c2e-4656-877e-ba884c88cf4d-console-serving-cert\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:55:43.009657 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:43.009621 2571 generic.go:356] "Generic (PLEG): container finished" podID="b0833a28-5c2e-4656-877e-ba884c88cf4d" containerID="477f57a1432568a0e0f4d0a64fcdd1817b9190f3e5f20ae759fe63f6f0874905" exitCode=0 Jul 16 16:55:43.010048 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:43.009671 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55f47d678-psdcp" Jul 16 16:55:43.010048 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:43.009705 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55f47d678-psdcp" event={"ID":"b0833a28-5c2e-4656-877e-ba884c88cf4d","Type":"ContainerDied","Data":"477f57a1432568a0e0f4d0a64fcdd1817b9190f3e5f20ae759fe63f6f0874905"} Jul 16 16:55:43.010048 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:43.009775 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55f47d678-psdcp" event={"ID":"b0833a28-5c2e-4656-877e-ba884c88cf4d","Type":"ContainerDied","Data":"63408288c1a89ee44d9809cda53295ec114fe4725b23305b8f36038f552a9947"} Jul 16 16:55:43.010048 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:43.009797 2571 scope.go:117] "RemoveContainer" containerID="477f57a1432568a0e0f4d0a64fcdd1817b9190f3e5f20ae759fe63f6f0874905" Jul 16 16:55:43.017699 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:43.017658 2571 scope.go:117] "RemoveContainer" containerID="477f57a1432568a0e0f4d0a64fcdd1817b9190f3e5f20ae759fe63f6f0874905" Jul 16 16:55:43.017966 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:55:43.017947 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"477f57a1432568a0e0f4d0a64fcdd1817b9190f3e5f20ae759fe63f6f0874905\": container with ID starting with 477f57a1432568a0e0f4d0a64fcdd1817b9190f3e5f20ae759fe63f6f0874905 not found: ID does not exist" containerID="477f57a1432568a0e0f4d0a64fcdd1817b9190f3e5f20ae759fe63f6f0874905" Jul 16 16:55:43.018070 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:43.017974 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"477f57a1432568a0e0f4d0a64fcdd1817b9190f3e5f20ae759fe63f6f0874905"} err="failed to get container status \"477f57a1432568a0e0f4d0a64fcdd1817b9190f3e5f20ae759fe63f6f0874905\": rpc error: code = NotFound desc = could not find container \"477f57a1432568a0e0f4d0a64fcdd1817b9190f3e5f20ae759fe63f6f0874905\": container with ID starting with 477f57a1432568a0e0f4d0a64fcdd1817b9190f3e5f20ae759fe63f6f0874905 not found: ID does not exist" Jul 16 16:55:43.028310 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:43.028287 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55f47d678-psdcp"] Jul 16 16:55:43.033649 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:43.033594 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-55f47d678-psdcp"] Jul 16 16:55:44.776702 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:44.776665 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0833a28-5c2e-4656-877e-ba884c88cf4d" path="/var/lib/kubelet/pods/b0833a28-5c2e-4656-877e-ba884c88cf4d/volumes" Jul 16 16:55:48.674022 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:48.673995 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r8cn4_f637d961-672d-4aa2-a49a-fd021c2d4df2/console-operator/2.log" Jul 16 16:55:48.676234 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:48.676213 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r8cn4_f637d961-672d-4aa2-a49a-fd021c2d4df2/console-operator/2.log" Jul 16 16:55:48.682835 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:48.682814 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qd7bg_cfbe9f0b-b759-4064-9332-3b48c547c003/ovn-acl-logging/0.log" Jul 16 16:55:48.685181 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:48.685142 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qd7bg_cfbe9f0b-b759-4064-9332-3b48c547c003/ovn-acl-logging/0.log" Jul 16 16:55:48.685328 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:55:48.685214 2571 kubelet.go:1648] "Image garbage collection succeeded" Jul 16 16:56:16.449452 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:16.449413 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wdn2b"] Jul 16 16:56:16.451892 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:16.449691 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0833a28-5c2e-4656-877e-ba884c88cf4d" containerName="console" Jul 16 16:56:16.451892 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:16.449702 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0833a28-5c2e-4656-877e-ba884c88cf4d" containerName="console" Jul 16 16:56:16.451892 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:16.449766 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0833a28-5c2e-4656-877e-ba884c88cf4d" containerName="console" Jul 16 16:56:16.452846 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:16.452829 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wdn2b" Jul 16 16:56:16.455561 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:16.455539 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 16 16:56:16.456601 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:16.456583 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 16 16:56:16.456680 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:16.456617 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-z49l7\"" Jul 16 16:56:16.462333 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:16.462309 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wdn2b"] Jul 16 16:56:16.561498 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:16.561464 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8blg\" (UniqueName: \"kubernetes.io/projected/d5d23349-3b22-4762-9410-c1081f9e21d2-kube-api-access-d8blg\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wdn2b\" (UID: \"d5d23349-3b22-4762-9410-c1081f9e21d2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wdn2b" Jul 16 16:56:16.561498 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:16.561499 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5d23349-3b22-4762-9410-c1081f9e21d2-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wdn2b\" (UID: \"d5d23349-3b22-4762-9410-c1081f9e21d2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wdn2b" Jul 16 16:56:16.561707 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:16.561519 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5d23349-3b22-4762-9410-c1081f9e21d2-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wdn2b\" (UID: \"d5d23349-3b22-4762-9410-c1081f9e21d2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wdn2b" Jul 16 16:56:16.662102 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:16.662063 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8blg\" (UniqueName: \"kubernetes.io/projected/d5d23349-3b22-4762-9410-c1081f9e21d2-kube-api-access-d8blg\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wdn2b\" (UID: \"d5d23349-3b22-4762-9410-c1081f9e21d2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wdn2b" Jul 16 16:56:16.662102 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:16.662103 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5d23349-3b22-4762-9410-c1081f9e21d2-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wdn2b\" (UID: \"d5d23349-3b22-4762-9410-c1081f9e21d2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wdn2b" Jul 16 16:56:16.662311 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:16.662122 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5d23349-3b22-4762-9410-c1081f9e21d2-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wdn2b\" (UID: \"d5d23349-3b22-4762-9410-c1081f9e21d2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wdn2b" Jul 16 16:56:16.662542 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:16.662523 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5d23349-3b22-4762-9410-c1081f9e21d2-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wdn2b\" (UID: \"d5d23349-3b22-4762-9410-c1081f9e21d2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wdn2b" Jul 16 16:56:16.662577 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:16.662530 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5d23349-3b22-4762-9410-c1081f9e21d2-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wdn2b\" (UID: \"d5d23349-3b22-4762-9410-c1081f9e21d2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wdn2b" Jul 16 16:56:16.674636 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:16.674612 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8blg\" (UniqueName: \"kubernetes.io/projected/d5d23349-3b22-4762-9410-c1081f9e21d2-kube-api-access-d8blg\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wdn2b\" (UID: \"d5d23349-3b22-4762-9410-c1081f9e21d2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wdn2b" Jul 16 16:56:16.762537 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:16.762454 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wdn2b" Jul 16 16:56:16.895947 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:16.895919 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wdn2b"] Jul 16 16:56:16.897998 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:56:16.897965 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5d23349_3b22_4762_9410_c1081f9e21d2.slice/crio-293490f30f6e7dafc5e8a506c2e0ee76b1e6bdd4dd7af1e0815a8033426e2bb3 WatchSource:0}: Error finding container 293490f30f6e7dafc5e8a506c2e0ee76b1e6bdd4dd7af1e0815a8033426e2bb3: Status 404 returned error can't find the container with id 293490f30f6e7dafc5e8a506c2e0ee76b1e6bdd4dd7af1e0815a8033426e2bb3 Jul 16 16:56:16.899953 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:16.899937 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 16:56:17.141697 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:17.141616 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wdn2b" event={"ID":"d5d23349-3b22-4762-9410-c1081f9e21d2","Type":"ContainerStarted","Data":"293490f30f6e7dafc5e8a506c2e0ee76b1e6bdd4dd7af1e0815a8033426e2bb3"} Jul 16 16:56:23.166195 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:23.166169 2571 generic.go:356] "Generic (PLEG): container finished" podID="d5d23349-3b22-4762-9410-c1081f9e21d2" containerID="ccf7d8e22991abebf93222e79e91f66756b9a2b04f7221b2f7437ad76788c84d" exitCode=0 Jul 16 16:56:23.166531 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:23.166212 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wdn2b" event={"ID":"d5d23349-3b22-4762-9410-c1081f9e21d2","Type":"ContainerDied","Data":"ccf7d8e22991abebf93222e79e91f66756b9a2b04f7221b2f7437ad76788c84d"} Jul 16 16:56:26.185619 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:26.185583 2571 generic.go:356] "Generic (PLEG): container finished" podID="d5d23349-3b22-4762-9410-c1081f9e21d2" containerID="ba42465514d41120161a6986f5708452d7fef88e81cddfd7ddc2aa50c3d4d7ea" exitCode=0 Jul 16 16:56:26.186056 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:26.185633 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wdn2b" event={"ID":"d5d23349-3b22-4762-9410-c1081f9e21d2","Type":"ContainerDied","Data":"ba42465514d41120161a6986f5708452d7fef88e81cddfd7ddc2aa50c3d4d7ea"} Jul 16 16:56:33.216822 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:33.216782 2571 generic.go:356] "Generic (PLEG): container finished" podID="d5d23349-3b22-4762-9410-c1081f9e21d2" containerID="98e548e87123225dc604584de8865aae62976a98ab7cbd92ae911fe0208db278" exitCode=0 Jul 16 16:56:33.216822 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:33.216831 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wdn2b" event={"ID":"d5d23349-3b22-4762-9410-c1081f9e21d2","Type":"ContainerDied","Data":"98e548e87123225dc604584de8865aae62976a98ab7cbd92ae911fe0208db278"} Jul 16 16:56:34.346769 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:34.346747 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wdn2b" Jul 16 16:56:34.506530 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:34.506463 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5d23349-3b22-4762-9410-c1081f9e21d2-util\") pod \"d5d23349-3b22-4762-9410-c1081f9e21d2\" (UID: \"d5d23349-3b22-4762-9410-c1081f9e21d2\") " Jul 16 16:56:34.506530 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:34.506522 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5d23349-3b22-4762-9410-c1081f9e21d2-bundle\") pod \"d5d23349-3b22-4762-9410-c1081f9e21d2\" (UID: \"d5d23349-3b22-4762-9410-c1081f9e21d2\") " Jul 16 16:56:34.506677 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:34.506556 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d8blg\" (UniqueName: \"kubernetes.io/projected/d5d23349-3b22-4762-9410-c1081f9e21d2-kube-api-access-d8blg\") pod \"d5d23349-3b22-4762-9410-c1081f9e21d2\" (UID: \"d5d23349-3b22-4762-9410-c1081f9e21d2\") " Jul 16 16:56:34.507154 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:34.507125 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5d23349-3b22-4762-9410-c1081f9e21d2-bundle" (OuterVolumeSpecName: "bundle") pod "d5d23349-3b22-4762-9410-c1081f9e21d2" (UID: "d5d23349-3b22-4762-9410-c1081f9e21d2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:56:34.508639 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:34.508616 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5d23349-3b22-4762-9410-c1081f9e21d2-kube-api-access-d8blg" (OuterVolumeSpecName: "kube-api-access-d8blg") pod "d5d23349-3b22-4762-9410-c1081f9e21d2" (UID: "d5d23349-3b22-4762-9410-c1081f9e21d2"). InnerVolumeSpecName "kube-api-access-d8blg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:56:34.510984 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:34.510959 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5d23349-3b22-4762-9410-c1081f9e21d2-util" (OuterVolumeSpecName: "util") pod "d5d23349-3b22-4762-9410-c1081f9e21d2" (UID: "d5d23349-3b22-4762-9410-c1081f9e21d2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:56:34.607320 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:34.607295 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5d23349-3b22-4762-9410-c1081f9e21d2-bundle\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:56:34.607320 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:34.607318 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8blg\" (UniqueName: \"kubernetes.io/projected/d5d23349-3b22-4762-9410-c1081f9e21d2-kube-api-access-d8blg\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:56:34.607445 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:34.607328 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5d23349-3b22-4762-9410-c1081f9e21d2-util\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:56:35.227289 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:35.227262 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wdn2b" Jul 16 16:56:35.227466 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:35.227265 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wdn2b" event={"ID":"d5d23349-3b22-4762-9410-c1081f9e21d2","Type":"ContainerDied","Data":"293490f30f6e7dafc5e8a506c2e0ee76b1e6bdd4dd7af1e0815a8033426e2bb3"} Jul 16 16:56:35.227466 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:35.227385 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="293490f30f6e7dafc5e8a506c2e0ee76b1e6bdd4dd7af1e0815a8033426e2bb3" Jul 16 16:56:40.051665 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:40.051623 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vl547"] Jul 16 16:56:40.052090 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:40.051913 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5d23349-3b22-4762-9410-c1081f9e21d2" containerName="util" Jul 16 16:56:40.052090 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:40.051926 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5d23349-3b22-4762-9410-c1081f9e21d2" containerName="util" Jul 16 16:56:40.052090 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:40.051936 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5d23349-3b22-4762-9410-c1081f9e21d2" containerName="pull" Jul 16 16:56:40.052090 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:40.051941 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5d23349-3b22-4762-9410-c1081f9e21d2" containerName="pull" Jul 16 16:56:40.052090 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:40.051949 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5d23349-3b22-4762-9410-c1081f9e21d2" containerName="extract" Jul 16 16:56:40.052090 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:40.051955 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5d23349-3b22-4762-9410-c1081f9e21d2" containerName="extract" Jul 16 16:56:40.052090 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:40.052017 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5d23349-3b22-4762-9410-c1081f9e21d2" containerName="extract" Jul 16 16:56:40.101366 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:40.101334 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vl547"] Jul 16 16:56:40.101508 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:40.101421 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vl547" Jul 16 16:56:40.104436 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:40.104412 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 16 16:56:40.104586 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:40.104418 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 16 16:56:40.104586 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:40.104472 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-ht6mf\"" Jul 16 16:56:40.244389 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:40.244358 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zg8lx\" (UniqueName: \"kubernetes.io/projected/6b4b44c0-efe4-4182-b257-b2af71453f60-kube-api-access-zg8lx\") pod \"cert-manager-operator-controller-manager-55c444bf86-vl547\" (UID: \"6b4b44c0-efe4-4182-b257-b2af71453f60\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vl547" Jul 16 16:56:40.244511 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:40.244406 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6b4b44c0-efe4-4182-b257-b2af71453f60-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-vl547\" (UID: \"6b4b44c0-efe4-4182-b257-b2af71453f60\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vl547" Jul 16 16:56:40.344901 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:40.344851 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zg8lx\" (UniqueName: \"kubernetes.io/projected/6b4b44c0-efe4-4182-b257-b2af71453f60-kube-api-access-zg8lx\") pod \"cert-manager-operator-controller-manager-55c444bf86-vl547\" (UID: \"6b4b44c0-efe4-4182-b257-b2af71453f60\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vl547" Jul 16 16:56:40.344901 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:40.344893 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6b4b44c0-efe4-4182-b257-b2af71453f60-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-vl547\" (UID: \"6b4b44c0-efe4-4182-b257-b2af71453f60\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vl547" Jul 16 16:56:40.345216 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:40.345201 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6b4b44c0-efe4-4182-b257-b2af71453f60-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-vl547\" (UID: \"6b4b44c0-efe4-4182-b257-b2af71453f60\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vl547" Jul 16 16:56:40.361714 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:40.361692 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zg8lx\" (UniqueName: \"kubernetes.io/projected/6b4b44c0-efe4-4182-b257-b2af71453f60-kube-api-access-zg8lx\") pod \"cert-manager-operator-controller-manager-55c444bf86-vl547\" (UID: \"6b4b44c0-efe4-4182-b257-b2af71453f60\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vl547" Jul 16 16:56:40.410651 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:40.410629 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vl547" Jul 16 16:56:40.566953 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:40.566928 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vl547"] Jul 16 16:56:40.568245 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:56:40.568222 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b4b44c0_efe4_4182_b257_b2af71453f60.slice/crio-a899d8543f58061501f2f0fc7edbd9e329397c7a39f59b394bdb4ccf453b0de2 WatchSource:0}: Error finding container a899d8543f58061501f2f0fc7edbd9e329397c7a39f59b394bdb4ccf453b0de2: Status 404 returned error can't find the container with id a899d8543f58061501f2f0fc7edbd9e329397c7a39f59b394bdb4ccf453b0de2 Jul 16 16:56:41.252250 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:41.252217 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vl547" event={"ID":"6b4b44c0-efe4-4182-b257-b2af71453f60","Type":"ContainerStarted","Data":"a899d8543f58061501f2f0fc7edbd9e329397c7a39f59b394bdb4ccf453b0de2"} Jul 16 16:56:43.262357 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:43.262301 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vl547" event={"ID":"6b4b44c0-efe4-4182-b257-b2af71453f60","Type":"ContainerStarted","Data":"3cf7d1e755f8c38fe4e2fc6788ddf12dfafdff60bf54ccdab2a769c15963e134"} Jul 16 16:56:43.288941 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:43.288899 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vl547" podStartSLOduration=1.202098342 podStartE2EDuration="3.28888491s" totalImagesPullingTime="2.086786568s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:56:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:56:43.286970451 +0000 UTC m=+355.028545762" watchObservedRunningTime="2026-07-16 16:56:43.28888491 +0000 UTC m=+355.030460205" Jul 16 16:56:43.975736 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:43.975705 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkbm8"] Jul 16 16:56:43.990988 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:43.990967 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkbm8" Jul 16 16:56:43.996878 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:43.996852 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkbm8"] Jul 16 16:56:43.997190 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:43.997128 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 16 16:56:43.997287 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:43.997247 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 16 16:56:43.998306 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:43.998290 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-z49l7\"" Jul 16 16:56:44.071460 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:44.071437 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/61bd4a35-528b-4cd8-bd38-2b9eb53c3797-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkbm8\" (UID: \"61bd4a35-528b-4cd8-bd38-2b9eb53c3797\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkbm8" Jul 16 16:56:44.071561 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:44.071473 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsnsf\" (UniqueName: \"kubernetes.io/projected/61bd4a35-528b-4cd8-bd38-2b9eb53c3797-kube-api-access-fsnsf\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkbm8\" (UID: \"61bd4a35-528b-4cd8-bd38-2b9eb53c3797\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkbm8" Jul 16 16:56:44.071561 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:44.071493 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/61bd4a35-528b-4cd8-bd38-2b9eb53c3797-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkbm8\" (UID: \"61bd4a35-528b-4cd8-bd38-2b9eb53c3797\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkbm8" Jul 16 16:56:44.172732 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:44.172709 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/61bd4a35-528b-4cd8-bd38-2b9eb53c3797-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkbm8\" (UID: \"61bd4a35-528b-4cd8-bd38-2b9eb53c3797\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkbm8" Jul 16 16:56:44.172842 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:44.172767 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsnsf\" (UniqueName: \"kubernetes.io/projected/61bd4a35-528b-4cd8-bd38-2b9eb53c3797-kube-api-access-fsnsf\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkbm8\" (UID: \"61bd4a35-528b-4cd8-bd38-2b9eb53c3797\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkbm8" Jul 16 16:56:44.172842 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:44.172801 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/61bd4a35-528b-4cd8-bd38-2b9eb53c3797-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkbm8\" (UID: \"61bd4a35-528b-4cd8-bd38-2b9eb53c3797\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkbm8" Jul 16 16:56:44.173167 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:44.173128 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/61bd4a35-528b-4cd8-bd38-2b9eb53c3797-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkbm8\" (UID: \"61bd4a35-528b-4cd8-bd38-2b9eb53c3797\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkbm8" Jul 16 16:56:44.173207 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:44.173165 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/61bd4a35-528b-4cd8-bd38-2b9eb53c3797-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkbm8\" (UID: \"61bd4a35-528b-4cd8-bd38-2b9eb53c3797\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkbm8" Jul 16 16:56:44.187358 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:44.187336 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsnsf\" (UniqueName: \"kubernetes.io/projected/61bd4a35-528b-4cd8-bd38-2b9eb53c3797-kube-api-access-fsnsf\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkbm8\" (UID: \"61bd4a35-528b-4cd8-bd38-2b9eb53c3797\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkbm8" Jul 16 16:56:44.300830 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:44.300780 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkbm8" Jul 16 16:56:44.415765 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:44.415720 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-x7hnl"] Jul 16 16:56:44.447903 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:56:44.447873 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61bd4a35_528b_4cd8_bd38_2b9eb53c3797.slice/crio-d2a35547c50ca440acb54c22aabf93dff7002b55592822181a451d5c7e572c50 WatchSource:0}: Error finding container d2a35547c50ca440acb54c22aabf93dff7002b55592822181a451d5c7e572c50: Status 404 returned error can't find the container with id d2a35547c50ca440acb54c22aabf93dff7002b55592822181a451d5c7e572c50 Jul 16 16:56:44.452311 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:44.452290 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-x7hnl"] Jul 16 16:56:44.452388 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:44.452324 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkbm8"] Jul 16 16:56:44.452467 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:44.452448 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-x7hnl" Jul 16 16:56:44.455397 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:44.455380 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 16 16:56:44.456224 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:44.456207 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 16 16:56:44.575812 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:44.575786 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ef85e45f-01bc-45ae-80fd-a15091346658-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-x7hnl\" (UID: \"ef85e45f-01bc-45ae-80fd-a15091346658\") " pod="cert-manager/cert-manager-webhook-6486ffd997-x7hnl" Jul 16 16:56:44.575901 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:44.575824 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrh6m\" (UniqueName: \"kubernetes.io/projected/ef85e45f-01bc-45ae-80fd-a15091346658-kube-api-access-zrh6m\") pod \"cert-manager-webhook-6486ffd997-x7hnl\" (UID: \"ef85e45f-01bc-45ae-80fd-a15091346658\") " pod="cert-manager/cert-manager-webhook-6486ffd997-x7hnl" Jul 16 16:56:44.676469 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:44.676444 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ef85e45f-01bc-45ae-80fd-a15091346658-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-x7hnl\" (UID: \"ef85e45f-01bc-45ae-80fd-a15091346658\") " pod="cert-manager/cert-manager-webhook-6486ffd997-x7hnl" Jul 16 16:56:44.676563 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:44.676479 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrh6m\" (UniqueName: \"kubernetes.io/projected/ef85e45f-01bc-45ae-80fd-a15091346658-kube-api-access-zrh6m\") pod \"cert-manager-webhook-6486ffd997-x7hnl\" (UID: \"ef85e45f-01bc-45ae-80fd-a15091346658\") " pod="cert-manager/cert-manager-webhook-6486ffd997-x7hnl" Jul 16 16:56:44.687656 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:44.687629 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ef85e45f-01bc-45ae-80fd-a15091346658-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-x7hnl\" (UID: \"ef85e45f-01bc-45ae-80fd-a15091346658\") " pod="cert-manager/cert-manager-webhook-6486ffd997-x7hnl" Jul 16 16:56:44.688067 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:44.688048 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrh6m\" (UniqueName: \"kubernetes.io/projected/ef85e45f-01bc-45ae-80fd-a15091346658-kube-api-access-zrh6m\") pod \"cert-manager-webhook-6486ffd997-x7hnl\" (UID: \"ef85e45f-01bc-45ae-80fd-a15091346658\") " pod="cert-manager/cert-manager-webhook-6486ffd997-x7hnl" Jul 16 16:56:44.770695 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:44.770668 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-x7hnl" Jul 16 16:56:44.920285 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:44.920263 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-x7hnl"] Jul 16 16:56:44.922405 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:56:44.922377 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podef85e45f_01bc_45ae_80fd_a15091346658.slice/crio-2b78e76df08d4b56bf297aaa8bf586bb77e0318ab1d0520e343a2756137bc542 WatchSource:0}: Error finding container 2b78e76df08d4b56bf297aaa8bf586bb77e0318ab1d0520e343a2756137bc542: Status 404 returned error can't find the container with id 2b78e76df08d4b56bf297aaa8bf586bb77e0318ab1d0520e343a2756137bc542 Jul 16 16:56:45.272107 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:45.272077 2571 generic.go:356] "Generic (PLEG): container finished" podID="61bd4a35-528b-4cd8-bd38-2b9eb53c3797" containerID="d109fabde5b679084989596781cb487b1f1c1a8537287e45983b5500a4ec8c2d" exitCode=0 Jul 16 16:56:45.272219 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:45.272161 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkbm8" event={"ID":"61bd4a35-528b-4cd8-bd38-2b9eb53c3797","Type":"ContainerDied","Data":"d109fabde5b679084989596781cb487b1f1c1a8537287e45983b5500a4ec8c2d"} Jul 16 16:56:45.272219 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:45.272210 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkbm8" event={"ID":"61bd4a35-528b-4cd8-bd38-2b9eb53c3797","Type":"ContainerStarted","Data":"d2a35547c50ca440acb54c22aabf93dff7002b55592822181a451d5c7e572c50"} Jul 16 16:56:45.273480 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:45.273456 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-x7hnl" event={"ID":"ef85e45f-01bc-45ae-80fd-a15091346658","Type":"ContainerStarted","Data":"2b78e76df08d4b56bf297aaa8bf586bb77e0318ab1d0520e343a2756137bc542"} Jul 16 16:56:49.293902 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:49.293860 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-x7hnl" event={"ID":"ef85e45f-01bc-45ae-80fd-a15091346658","Type":"ContainerStarted","Data":"bc453e5d9e62c245a0fac1f506d687e85b7693cd03010f2e96698cd0567e1010"} Jul 16 16:56:49.294300 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:49.293937 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-x7hnl" Jul 16 16:56:49.295702 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:49.295677 2571 generic.go:356] "Generic (PLEG): container finished" podID="61bd4a35-528b-4cd8-bd38-2b9eb53c3797" containerID="62b0ae826de16234f216dbb29279ab8dedddf963cf2d63cb47a8d93d341ab377" exitCode=0 Jul 16 16:56:49.295806 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:49.295719 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkbm8" event={"ID":"61bd4a35-528b-4cd8-bd38-2b9eb53c3797","Type":"ContainerDied","Data":"62b0ae826de16234f216dbb29279ab8dedddf963cf2d63cb47a8d93d341ab377"} Jul 16 16:56:49.322896 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:49.322849 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-x7hnl" podStartSLOduration=1.957188149 podStartE2EDuration="5.322832134s" totalImagesPullingTime="3.365643985s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:56:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:56:49.322023993 +0000 UTC m=+361.063599290" watchObservedRunningTime="2026-07-16 16:56:49.322832134 +0000 UTC m=+361.064407430" Jul 16 16:56:50.302042 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:50.302010 2571 generic.go:356] "Generic (PLEG): container finished" podID="61bd4a35-528b-4cd8-bd38-2b9eb53c3797" containerID="059ee196403cbb8141b8b8d6cc52754cb9b2f412c3829629729717313cc907c3" exitCode=0 Jul 16 16:56:50.302426 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:50.302091 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkbm8" event={"ID":"61bd4a35-528b-4cd8-bd38-2b9eb53c3797","Type":"ContainerDied","Data":"059ee196403cbb8141b8b8d6cc52754cb9b2f412c3829629729717313cc907c3"} Jul 16 16:56:51.428917 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:51.428896 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkbm8" Jul 16 16:56:51.531052 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:51.531027 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/61bd4a35-528b-4cd8-bd38-2b9eb53c3797-util\") pod \"61bd4a35-528b-4cd8-bd38-2b9eb53c3797\" (UID: \"61bd4a35-528b-4cd8-bd38-2b9eb53c3797\") " Jul 16 16:56:51.531193 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:51.531064 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fsnsf\" (UniqueName: \"kubernetes.io/projected/61bd4a35-528b-4cd8-bd38-2b9eb53c3797-kube-api-access-fsnsf\") pod \"61bd4a35-528b-4cd8-bd38-2b9eb53c3797\" (UID: \"61bd4a35-528b-4cd8-bd38-2b9eb53c3797\") " Jul 16 16:56:51.531193 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:51.531093 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/61bd4a35-528b-4cd8-bd38-2b9eb53c3797-bundle\") pod \"61bd4a35-528b-4cd8-bd38-2b9eb53c3797\" (UID: \"61bd4a35-528b-4cd8-bd38-2b9eb53c3797\") " Jul 16 16:56:51.531484 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:51.531459 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61bd4a35-528b-4cd8-bd38-2b9eb53c3797-bundle" (OuterVolumeSpecName: "bundle") pod "61bd4a35-528b-4cd8-bd38-2b9eb53c3797" (UID: "61bd4a35-528b-4cd8-bd38-2b9eb53c3797"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:56:51.533025 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:51.532998 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61bd4a35-528b-4cd8-bd38-2b9eb53c3797-kube-api-access-fsnsf" (OuterVolumeSpecName: "kube-api-access-fsnsf") pod "61bd4a35-528b-4cd8-bd38-2b9eb53c3797" (UID: "61bd4a35-528b-4cd8-bd38-2b9eb53c3797"). InnerVolumeSpecName "kube-api-access-fsnsf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:56:51.535218 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:51.535189 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61bd4a35-528b-4cd8-bd38-2b9eb53c3797-util" (OuterVolumeSpecName: "util") pod "61bd4a35-528b-4cd8-bd38-2b9eb53c3797" (UID: "61bd4a35-528b-4cd8-bd38-2b9eb53c3797"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:56:51.632502 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:51.632451 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/61bd4a35-528b-4cd8-bd38-2b9eb53c3797-util\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:56:51.632502 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:51.632472 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fsnsf\" (UniqueName: \"kubernetes.io/projected/61bd4a35-528b-4cd8-bd38-2b9eb53c3797-kube-api-access-fsnsf\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:56:51.632502 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:51.632481 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/61bd4a35-528b-4cd8-bd38-2b9eb53c3797-bundle\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:56:52.312907 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:52.312870 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkbm8" event={"ID":"61bd4a35-528b-4cd8-bd38-2b9eb53c3797","Type":"ContainerDied","Data":"d2a35547c50ca440acb54c22aabf93dff7002b55592822181a451d5c7e572c50"} Jul 16 16:56:52.313060 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:52.312927 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkbm8" Jul 16 16:56:52.313060 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:52.312931 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d2a35547c50ca440acb54c22aabf93dff7002b55592822181a451d5c7e572c50" Jul 16 16:56:55.304019 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:55.303990 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-x7hnl" Jul 16 16:56:58.428002 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:58.427966 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-n554m"] Jul 16 16:56:58.428372 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:58.428264 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61bd4a35-528b-4cd8-bd38-2b9eb53c3797" containerName="extract" Jul 16 16:56:58.428372 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:58.428275 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="61bd4a35-528b-4cd8-bd38-2b9eb53c3797" containerName="extract" Jul 16 16:56:58.428372 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:58.428286 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61bd4a35-528b-4cd8-bd38-2b9eb53c3797" containerName="pull" Jul 16 16:56:58.428372 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:58.428292 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="61bd4a35-528b-4cd8-bd38-2b9eb53c3797" containerName="pull" Jul 16 16:56:58.428372 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:58.428303 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61bd4a35-528b-4cd8-bd38-2b9eb53c3797" containerName="util" Jul 16 16:56:58.428372 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:58.428311 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="61bd4a35-528b-4cd8-bd38-2b9eb53c3797" containerName="util" Jul 16 16:56:58.428372 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:58.428363 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="61bd4a35-528b-4cd8-bd38-2b9eb53c3797" containerName="extract" Jul 16 16:56:58.431628 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:58.431610 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-n554m" Jul 16 16:56:58.434322 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:58.434303 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-7rh29\"" Jul 16 16:56:58.442522 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:58.442501 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-n554m"] Jul 16 16:56:58.584429 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:58.584392 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/327d2a84-68f0-4eb0-94ca-fe24947959e8-bound-sa-token\") pod \"cert-manager-55b4d954d5-n554m\" (UID: \"327d2a84-68f0-4eb0-94ca-fe24947959e8\") " pod="cert-manager/cert-manager-55b4d954d5-n554m" Jul 16 16:56:58.584607 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:58.584455 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dltkl\" (UniqueName: \"kubernetes.io/projected/327d2a84-68f0-4eb0-94ca-fe24947959e8-kube-api-access-dltkl\") pod \"cert-manager-55b4d954d5-n554m\" (UID: \"327d2a84-68f0-4eb0-94ca-fe24947959e8\") " pod="cert-manager/cert-manager-55b4d954d5-n554m" Jul 16 16:56:58.685067 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:58.684988 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dltkl\" (UniqueName: \"kubernetes.io/projected/327d2a84-68f0-4eb0-94ca-fe24947959e8-kube-api-access-dltkl\") pod \"cert-manager-55b4d954d5-n554m\" (UID: \"327d2a84-68f0-4eb0-94ca-fe24947959e8\") " pod="cert-manager/cert-manager-55b4d954d5-n554m" Jul 16 16:56:58.685067 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:58.685045 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/327d2a84-68f0-4eb0-94ca-fe24947959e8-bound-sa-token\") pod \"cert-manager-55b4d954d5-n554m\" (UID: \"327d2a84-68f0-4eb0-94ca-fe24947959e8\") " pod="cert-manager/cert-manager-55b4d954d5-n554m" Jul 16 16:56:58.701101 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:58.701068 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dltkl\" (UniqueName: \"kubernetes.io/projected/327d2a84-68f0-4eb0-94ca-fe24947959e8-kube-api-access-dltkl\") pod \"cert-manager-55b4d954d5-n554m\" (UID: \"327d2a84-68f0-4eb0-94ca-fe24947959e8\") " pod="cert-manager/cert-manager-55b4d954d5-n554m" Jul 16 16:56:58.701325 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:58.701307 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/327d2a84-68f0-4eb0-94ca-fe24947959e8-bound-sa-token\") pod \"cert-manager-55b4d954d5-n554m\" (UID: \"327d2a84-68f0-4eb0-94ca-fe24947959e8\") " pod="cert-manager/cert-manager-55b4d954d5-n554m" Jul 16 16:56:58.740246 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:58.740224 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-n554m" Jul 16 16:56:58.872910 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:58.872882 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-n554m"] Jul 16 16:56:58.876228 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:56:58.876191 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod327d2a84_68f0_4eb0_94ca_fe24947959e8.slice/crio-9e97c61712b2dad599d451bf098c030eb11f43acc87af3f722e2b5b89305caae WatchSource:0}: Error finding container 9e97c61712b2dad599d451bf098c030eb11f43acc87af3f722e2b5b89305caae: Status 404 returned error can't find the container with id 9e97c61712b2dad599d451bf098c030eb11f43acc87af3f722e2b5b89305caae Jul 16 16:56:59.343248 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:59.343209 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-n554m" event={"ID":"327d2a84-68f0-4eb0-94ca-fe24947959e8","Type":"ContainerStarted","Data":"698f0be0b5e74fd901bba47407fc843f5c67202c7c1156fe6d7b8266e40d59f4"} Jul 16 16:56:59.343248 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:59.343251 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-n554m" event={"ID":"327d2a84-68f0-4eb0-94ca-fe24947959e8","Type":"ContainerStarted","Data":"9e97c61712b2dad599d451bf098c030eb11f43acc87af3f722e2b5b89305caae"} Jul 16 16:56:59.363375 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:56:59.363321 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-n554m" podStartSLOduration=1.363307415 podStartE2EDuration="1.363307415s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:56:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:56:59.361415555 +0000 UTC m=+371.102990853" watchObservedRunningTime="2026-07-16 16:56:59.363307415 +0000 UTC m=+371.104882711" Jul 16 16:57:03.259267 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:03.259228 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsq5kq"] Jul 16 16:57:03.262816 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:03.262795 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsq5kq" Jul 16 16:57:03.267362 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:03.267330 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 16 16:57:03.268406 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:03.268387 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 16 16:57:03.268521 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:03.268410 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-z49l7\"" Jul 16 16:57:03.273146 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:03.273110 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsq5kq"] Jul 16 16:57:03.324050 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:03.324027 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1c55c013-7df8-49af-ab31-e6ff228b14d8-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsq5kq\" (UID: \"1c55c013-7df8-49af-ab31-e6ff228b14d8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsq5kq" Jul 16 16:57:03.324143 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:03.324097 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94wdp\" (UniqueName: \"kubernetes.io/projected/1c55c013-7df8-49af-ab31-e6ff228b14d8-kube-api-access-94wdp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsq5kq\" (UID: \"1c55c013-7df8-49af-ab31-e6ff228b14d8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsq5kq" Jul 16 16:57:03.324143 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:03.324135 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1c55c013-7df8-49af-ab31-e6ff228b14d8-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsq5kq\" (UID: \"1c55c013-7df8-49af-ab31-e6ff228b14d8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsq5kq" Jul 16 16:57:03.424617 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:03.424592 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94wdp\" (UniqueName: \"kubernetes.io/projected/1c55c013-7df8-49af-ab31-e6ff228b14d8-kube-api-access-94wdp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsq5kq\" (UID: \"1c55c013-7df8-49af-ab31-e6ff228b14d8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsq5kq" Jul 16 16:57:03.424715 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:03.424628 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1c55c013-7df8-49af-ab31-e6ff228b14d8-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsq5kq\" (UID: \"1c55c013-7df8-49af-ab31-e6ff228b14d8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsq5kq" Jul 16 16:57:03.424715 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:03.424667 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1c55c013-7df8-49af-ab31-e6ff228b14d8-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsq5kq\" (UID: \"1c55c013-7df8-49af-ab31-e6ff228b14d8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsq5kq" Jul 16 16:57:03.425011 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:03.424997 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1c55c013-7df8-49af-ab31-e6ff228b14d8-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsq5kq\" (UID: \"1c55c013-7df8-49af-ab31-e6ff228b14d8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsq5kq" Jul 16 16:57:03.425047 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:03.425022 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1c55c013-7df8-49af-ab31-e6ff228b14d8-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsq5kq\" (UID: \"1c55c013-7df8-49af-ab31-e6ff228b14d8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsq5kq" Jul 16 16:57:03.436550 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:03.436526 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94wdp\" (UniqueName: \"kubernetes.io/projected/1c55c013-7df8-49af-ab31-e6ff228b14d8-kube-api-access-94wdp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsq5kq\" (UID: \"1c55c013-7df8-49af-ab31-e6ff228b14d8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsq5kq" Jul 16 16:57:03.573633 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:03.573567 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsq5kq" Jul 16 16:57:03.701312 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:03.701286 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsq5kq"] Jul 16 16:57:03.702946 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:57:03.702915 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c55c013_7df8_49af_ab31_e6ff228b14d8.slice/crio-8fc570d5762f7ca889f0a90a5fd7b6a5f2a2cf66104322c4869015c5517a4688 WatchSource:0}: Error finding container 8fc570d5762f7ca889f0a90a5fd7b6a5f2a2cf66104322c4869015c5517a4688: Status 404 returned error can't find the container with id 8fc570d5762f7ca889f0a90a5fd7b6a5f2a2cf66104322c4869015c5517a4688 Jul 16 16:57:04.366563 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:04.366527 2571 generic.go:356] "Generic (PLEG): container finished" podID="1c55c013-7df8-49af-ab31-e6ff228b14d8" containerID="fbace77a4c45a07c6da298c9a86cd903fb8b488e11e5fed42ad1ddf061aa8100" exitCode=0 Jul 16 16:57:04.366977 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:04.366574 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsq5kq" event={"ID":"1c55c013-7df8-49af-ab31-e6ff228b14d8","Type":"ContainerDied","Data":"fbace77a4c45a07c6da298c9a86cd903fb8b488e11e5fed42ad1ddf061aa8100"} Jul 16 16:57:04.366977 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:04.366627 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsq5kq" event={"ID":"1c55c013-7df8-49af-ab31-e6ff228b14d8","Type":"ContainerStarted","Data":"8fc570d5762f7ca889f0a90a5fd7b6a5f2a2cf66104322c4869015c5517a4688"} Jul 16 16:57:05.372699 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:05.372662 2571 generic.go:356] "Generic (PLEG): container finished" podID="1c55c013-7df8-49af-ab31-e6ff228b14d8" containerID="ea330e1251787edcfa1000a1e14031c570ea554f869196dc525539138e0f4735" exitCode=0 Jul 16 16:57:05.373075 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:05.372767 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsq5kq" event={"ID":"1c55c013-7df8-49af-ab31-e6ff228b14d8","Type":"ContainerDied","Data":"ea330e1251787edcfa1000a1e14031c570ea554f869196dc525539138e0f4735"} Jul 16 16:57:06.379079 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:06.379039 2571 generic.go:356] "Generic (PLEG): container finished" podID="1c55c013-7df8-49af-ab31-e6ff228b14d8" containerID="561fd018230f66ce76bd855b57b062a59a935052921426476b51d18fed7bdc78" exitCode=0 Jul 16 16:57:06.379441 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:06.379086 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsq5kq" event={"ID":"1c55c013-7df8-49af-ab31-e6ff228b14d8","Type":"ContainerDied","Data":"561fd018230f66ce76bd855b57b062a59a935052921426476b51d18fed7bdc78"} Jul 16 16:57:07.502140 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:07.502113 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsq5kq" Jul 16 16:57:07.557350 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:07.557305 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1c55c013-7df8-49af-ab31-e6ff228b14d8-util\") pod \"1c55c013-7df8-49af-ab31-e6ff228b14d8\" (UID: \"1c55c013-7df8-49af-ab31-e6ff228b14d8\") " Jul 16 16:57:07.557517 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:07.557401 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1c55c013-7df8-49af-ab31-e6ff228b14d8-bundle\") pod \"1c55c013-7df8-49af-ab31-e6ff228b14d8\" (UID: \"1c55c013-7df8-49af-ab31-e6ff228b14d8\") " Jul 16 16:57:07.557517 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:07.557462 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-94wdp\" (UniqueName: \"kubernetes.io/projected/1c55c013-7df8-49af-ab31-e6ff228b14d8-kube-api-access-94wdp\") pod \"1c55c013-7df8-49af-ab31-e6ff228b14d8\" (UID: \"1c55c013-7df8-49af-ab31-e6ff228b14d8\") " Jul 16 16:57:07.558173 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:07.558144 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c55c013-7df8-49af-ab31-e6ff228b14d8-bundle" (OuterVolumeSpecName: "bundle") pod "1c55c013-7df8-49af-ab31-e6ff228b14d8" (UID: "1c55c013-7df8-49af-ab31-e6ff228b14d8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:57:07.559532 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:07.559503 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c55c013-7df8-49af-ab31-e6ff228b14d8-kube-api-access-94wdp" (OuterVolumeSpecName: "kube-api-access-94wdp") pod "1c55c013-7df8-49af-ab31-e6ff228b14d8" (UID: "1c55c013-7df8-49af-ab31-e6ff228b14d8"). InnerVolumeSpecName "kube-api-access-94wdp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:57:07.562717 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:07.562685 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c55c013-7df8-49af-ab31-e6ff228b14d8-util" (OuterVolumeSpecName: "util") pod "1c55c013-7df8-49af-ab31-e6ff228b14d8" (UID: "1c55c013-7df8-49af-ab31-e6ff228b14d8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:57:07.658404 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:07.658315 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94wdp\" (UniqueName: \"kubernetes.io/projected/1c55c013-7df8-49af-ab31-e6ff228b14d8-kube-api-access-94wdp\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:57:07.658404 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:07.658344 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1c55c013-7df8-49af-ab31-e6ff228b14d8-util\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:57:07.658404 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:07.658368 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1c55c013-7df8-49af-ab31-e6ff228b14d8-bundle\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:57:08.389441 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:08.389415 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsq5kq" Jul 16 16:57:08.389441 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:08.389432 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsq5kq" event={"ID":"1c55c013-7df8-49af-ab31-e6ff228b14d8","Type":"ContainerDied","Data":"8fc570d5762f7ca889f0a90a5fd7b6a5f2a2cf66104322c4869015c5517a4688"} Jul 16 16:57:08.389649 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:08.389471 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8fc570d5762f7ca889f0a90a5fd7b6a5f2a2cf66104322c4869015c5517a4688" Jul 16 16:57:16.550741 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:16.550692 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c7f465c76-fwvtz"] Jul 16 16:57:16.551154 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:16.550992 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c55c013-7df8-49af-ab31-e6ff228b14d8" containerName="util" Jul 16 16:57:16.551154 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:16.551004 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c55c013-7df8-49af-ab31-e6ff228b14d8" containerName="util" Jul 16 16:57:16.551154 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:16.551022 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c55c013-7df8-49af-ab31-e6ff228b14d8" containerName="pull" Jul 16 16:57:16.551154 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:16.551028 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c55c013-7df8-49af-ab31-e6ff228b14d8" containerName="pull" Jul 16 16:57:16.551154 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:16.551038 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c55c013-7df8-49af-ab31-e6ff228b14d8" containerName="extract" Jul 16 16:57:16.551154 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:16.551046 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c55c013-7df8-49af-ab31-e6ff228b14d8" containerName="extract" Jul 16 16:57:16.551154 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:16.551110 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c55c013-7df8-49af-ab31-e6ff228b14d8" containerName="extract" Jul 16 16:57:16.555239 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:16.555221 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-c7f465c76-fwvtz" Jul 16 16:57:16.557963 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:16.557929 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 16 16:57:16.558145 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:16.558104 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 16 16:57:16.558316 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:16.558297 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-sdg7k\"" Jul 16 16:57:16.558408 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:16.558392 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 16 16:57:16.558484 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:16.558466 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 16 16:57:16.575775 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:16.575754 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c7f465c76-fwvtz"] Jul 16 16:57:16.623806 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:16.623784 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d4b8d3c4-3c0c-4fb9-9ad0-a3af9bd3495f-manifests\") pod \"opendatahub-operator-controller-manager-c7f465c76-fwvtz\" (UID: \"d4b8d3c4-3c0c-4fb9-9ad0-a3af9bd3495f\") " pod="opendatahub/opendatahub-operator-controller-manager-c7f465c76-fwvtz" Jul 16 16:57:16.623882 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:16.623822 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48xzb\" (UniqueName: \"kubernetes.io/projected/d4b8d3c4-3c0c-4fb9-9ad0-a3af9bd3495f-kube-api-access-48xzb\") pod \"opendatahub-operator-controller-manager-c7f465c76-fwvtz\" (UID: \"d4b8d3c4-3c0c-4fb9-9ad0-a3af9bd3495f\") " pod="opendatahub/opendatahub-operator-controller-manager-c7f465c76-fwvtz" Jul 16 16:57:16.623882 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:16.623848 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d4b8d3c4-3c0c-4fb9-9ad0-a3af9bd3495f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c7f465c76-fwvtz\" (UID: \"d4b8d3c4-3c0c-4fb9-9ad0-a3af9bd3495f\") " pod="opendatahub/opendatahub-operator-controller-manager-c7f465c76-fwvtz" Jul 16 16:57:16.623967 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:16.623900 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d4b8d3c4-3c0c-4fb9-9ad0-a3af9bd3495f-webhook-cert\") pod \"opendatahub-operator-controller-manager-c7f465c76-fwvtz\" (UID: \"d4b8d3c4-3c0c-4fb9-9ad0-a3af9bd3495f\") " pod="opendatahub/opendatahub-operator-controller-manager-c7f465c76-fwvtz" Jul 16 16:57:16.623967 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:16.623923 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d4b8d3c4-3c0c-4fb9-9ad0-a3af9bd3495f-tmp\") pod \"opendatahub-operator-controller-manager-c7f465c76-fwvtz\" (UID: \"d4b8d3c4-3c0c-4fb9-9ad0-a3af9bd3495f\") " pod="opendatahub/opendatahub-operator-controller-manager-c7f465c76-fwvtz" Jul 16 16:57:16.725199 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:16.725175 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d4b8d3c4-3c0c-4fb9-9ad0-a3af9bd3495f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c7f465c76-fwvtz\" (UID: \"d4b8d3c4-3c0c-4fb9-9ad0-a3af9bd3495f\") " pod="opendatahub/opendatahub-operator-controller-manager-c7f465c76-fwvtz" Jul 16 16:57:16.725287 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:16.725218 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d4b8d3c4-3c0c-4fb9-9ad0-a3af9bd3495f-webhook-cert\") pod \"opendatahub-operator-controller-manager-c7f465c76-fwvtz\" (UID: \"d4b8d3c4-3c0c-4fb9-9ad0-a3af9bd3495f\") " pod="opendatahub/opendatahub-operator-controller-manager-c7f465c76-fwvtz" Jul 16 16:57:16.725287 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:16.725268 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d4b8d3c4-3c0c-4fb9-9ad0-a3af9bd3495f-tmp\") pod \"opendatahub-operator-controller-manager-c7f465c76-fwvtz\" (UID: \"d4b8d3c4-3c0c-4fb9-9ad0-a3af9bd3495f\") " pod="opendatahub/opendatahub-operator-controller-manager-c7f465c76-fwvtz" Jul 16 16:57:16.725379 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:16.725306 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d4b8d3c4-3c0c-4fb9-9ad0-a3af9bd3495f-manifests\") pod \"opendatahub-operator-controller-manager-c7f465c76-fwvtz\" (UID: \"d4b8d3c4-3c0c-4fb9-9ad0-a3af9bd3495f\") " pod="opendatahub/opendatahub-operator-controller-manager-c7f465c76-fwvtz" Jul 16 16:57:16.725379 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:16.725346 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48xzb\" (UniqueName: \"kubernetes.io/projected/d4b8d3c4-3c0c-4fb9-9ad0-a3af9bd3495f-kube-api-access-48xzb\") pod \"opendatahub-operator-controller-manager-c7f465c76-fwvtz\" (UID: \"d4b8d3c4-3c0c-4fb9-9ad0-a3af9bd3495f\") " pod="opendatahub/opendatahub-operator-controller-manager-c7f465c76-fwvtz" Jul 16 16:57:16.725767 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:16.725722 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d4b8d3c4-3c0c-4fb9-9ad0-a3af9bd3495f-manifests\") pod \"opendatahub-operator-controller-manager-c7f465c76-fwvtz\" (UID: \"d4b8d3c4-3c0c-4fb9-9ad0-a3af9bd3495f\") " pod="opendatahub/opendatahub-operator-controller-manager-c7f465c76-fwvtz" Jul 16 16:57:16.727635 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:16.727613 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d4b8d3c4-3c0c-4fb9-9ad0-a3af9bd3495f-tmp\") pod \"opendatahub-operator-controller-manager-c7f465c76-fwvtz\" (UID: \"d4b8d3c4-3c0c-4fb9-9ad0-a3af9bd3495f\") " pod="opendatahub/opendatahub-operator-controller-manager-c7f465c76-fwvtz" Jul 16 16:57:16.727761 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:16.727654 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d4b8d3c4-3c0c-4fb9-9ad0-a3af9bd3495f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c7f465c76-fwvtz\" (UID: \"d4b8d3c4-3c0c-4fb9-9ad0-a3af9bd3495f\") " pod="opendatahub/opendatahub-operator-controller-manager-c7f465c76-fwvtz" Jul 16 16:57:16.727839 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:16.727804 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d4b8d3c4-3c0c-4fb9-9ad0-a3af9bd3495f-webhook-cert\") pod \"opendatahub-operator-controller-manager-c7f465c76-fwvtz\" (UID: \"d4b8d3c4-3c0c-4fb9-9ad0-a3af9bd3495f\") " pod="opendatahub/opendatahub-operator-controller-manager-c7f465c76-fwvtz" Jul 16 16:57:16.735331 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:16.735314 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48xzb\" (UniqueName: \"kubernetes.io/projected/d4b8d3c4-3c0c-4fb9-9ad0-a3af9bd3495f-kube-api-access-48xzb\") pod \"opendatahub-operator-controller-manager-c7f465c76-fwvtz\" (UID: \"d4b8d3c4-3c0c-4fb9-9ad0-a3af9bd3495f\") " pod="opendatahub/opendatahub-operator-controller-manager-c7f465c76-fwvtz" Jul 16 16:57:16.864888 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:16.864831 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-c7f465c76-fwvtz" Jul 16 16:57:17.019693 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:57:17.019657 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4b8d3c4_3c0c_4fb9_9ad0_a3af9bd3495f.slice/crio-6c0b983846bb8eec7da5c899beed7b626dc5afa858a8fd5184085fb2a8e4c2f0 WatchSource:0}: Error finding container 6c0b983846bb8eec7da5c899beed7b626dc5afa858a8fd5184085fb2a8e4c2f0: Status 404 returned error can't find the container with id 6c0b983846bb8eec7da5c899beed7b626dc5afa858a8fd5184085fb2a8e4c2f0 Jul 16 16:57:17.023753 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:17.023713 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c7f465c76-fwvtz"] Jul 16 16:57:17.431947 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:17.431911 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c7f465c76-fwvtz" event={"ID":"d4b8d3c4-3c0c-4fb9-9ad0-a3af9bd3495f","Type":"ContainerStarted","Data":"6c0b983846bb8eec7da5c899beed7b626dc5afa858a8fd5184085fb2a8e4c2f0"} Jul 16 16:57:20.449641 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:20.449600 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c7f465c76-fwvtz" event={"ID":"d4b8d3c4-3c0c-4fb9-9ad0-a3af9bd3495f","Type":"ContainerStarted","Data":"14a21663738aa41e2687c46a8bd849f7a33ba969b7b92bc7cd7832bdd7af990e"} Jul 16 16:57:21.456291 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:21.456212 2571 generic.go:356] "Generic (PLEG): container finished" podID="d4b8d3c4-3c0c-4fb9-9ad0-a3af9bd3495f" containerID="14a21663738aa41e2687c46a8bd849f7a33ba969b7b92bc7cd7832bdd7af990e" exitCode=0 Jul 16 16:57:21.456698 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:21.456304 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c7f465c76-fwvtz" event={"ID":"d4b8d3c4-3c0c-4fb9-9ad0-a3af9bd3495f","Type":"ContainerDied","Data":"14a21663738aa41e2687c46a8bd849f7a33ba969b7b92bc7cd7832bdd7af990e"} Jul 16 16:57:22.463946 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:22.463900 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c7f465c76-fwvtz" event={"ID":"d4b8d3c4-3c0c-4fb9-9ad0-a3af9bd3495f","Type":"ContainerStarted","Data":"3a57394b249d86850c781ae43485568de9982594342d2acbaa234d3389e7146a"} Jul 16 16:57:22.464375 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:22.464019 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-c7f465c76-fwvtz" Jul 16 16:57:22.504238 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:22.504190 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-c7f465c76-fwvtz" podStartSLOduration=1.971410046 podStartE2EDuration="6.50417427s" totalImagesPullingTime="3.439458131s" totalInitContainerRuntime="1.093306093s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:57:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:57:22.501892004 +0000 UTC m=+394.243467301" watchObservedRunningTime="2026-07-16 16:57:22.50417427 +0000 UTC m=+394.245749563" Jul 16 16:57:33.470064 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:33.470032 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-c7f465c76-fwvtz" Jul 16 16:57:34.008826 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:34.008794 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7c59ccc8d7-mgk5b"] Jul 16 16:57:34.017153 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:34.017126 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7c59ccc8d7-mgk5b" Jul 16 16:57:34.021809 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:34.021769 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 16 16:57:34.021942 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:34.021815 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-6v6ql\"" Jul 16 16:57:34.021942 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:34.021862 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 16 16:57:34.022399 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:34.022374 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7c59ccc8d7-mgk5b"] Jul 16 16:57:34.022514 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:34.022502 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 16 16:57:34.022580 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:34.022511 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 16 16:57:34.022635 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:34.022511 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 16 16:57:34.148706 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:34.148683 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e651a638-235f-4c0e-8867-2c1eeace9b9d-manager-config\") pod \"lws-controller-manager-7c59ccc8d7-mgk5b\" (UID: \"e651a638-235f-4c0e-8867-2c1eeace9b9d\") " pod="openshift-lws-operator/lws-controller-manager-7c59ccc8d7-mgk5b" Jul 16 16:57:34.148850 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:34.148710 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e651a638-235f-4c0e-8867-2c1eeace9b9d-cert\") pod \"lws-controller-manager-7c59ccc8d7-mgk5b\" (UID: \"e651a638-235f-4c0e-8867-2c1eeace9b9d\") " pod="openshift-lws-operator/lws-controller-manager-7c59ccc8d7-mgk5b" Jul 16 16:57:34.148850 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:34.148760 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnpsr\" (UniqueName: \"kubernetes.io/projected/e651a638-235f-4c0e-8867-2c1eeace9b9d-kube-api-access-xnpsr\") pod \"lws-controller-manager-7c59ccc8d7-mgk5b\" (UID: \"e651a638-235f-4c0e-8867-2c1eeace9b9d\") " pod="openshift-lws-operator/lws-controller-manager-7c59ccc8d7-mgk5b" Jul 16 16:57:34.148967 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:34.148900 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e651a638-235f-4c0e-8867-2c1eeace9b9d-metrics-cert\") pod \"lws-controller-manager-7c59ccc8d7-mgk5b\" (UID: \"e651a638-235f-4c0e-8867-2c1eeace9b9d\") " pod="openshift-lws-operator/lws-controller-manager-7c59ccc8d7-mgk5b" Jul 16 16:57:34.250117 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:34.250091 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e651a638-235f-4c0e-8867-2c1eeace9b9d-metrics-cert\") pod \"lws-controller-manager-7c59ccc8d7-mgk5b\" (UID: \"e651a638-235f-4c0e-8867-2c1eeace9b9d\") " pod="openshift-lws-operator/lws-controller-manager-7c59ccc8d7-mgk5b" Jul 16 16:57:34.250217 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:34.250123 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e651a638-235f-4c0e-8867-2c1eeace9b9d-manager-config\") pod \"lws-controller-manager-7c59ccc8d7-mgk5b\" (UID: \"e651a638-235f-4c0e-8867-2c1eeace9b9d\") " pod="openshift-lws-operator/lws-controller-manager-7c59ccc8d7-mgk5b" Jul 16 16:57:34.250217 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:34.250141 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e651a638-235f-4c0e-8867-2c1eeace9b9d-cert\") pod \"lws-controller-manager-7c59ccc8d7-mgk5b\" (UID: \"e651a638-235f-4c0e-8867-2c1eeace9b9d\") " pod="openshift-lws-operator/lws-controller-manager-7c59ccc8d7-mgk5b" Jul 16 16:57:34.250217 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:34.250170 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnpsr\" (UniqueName: \"kubernetes.io/projected/e651a638-235f-4c0e-8867-2c1eeace9b9d-kube-api-access-xnpsr\") pod \"lws-controller-manager-7c59ccc8d7-mgk5b\" (UID: \"e651a638-235f-4c0e-8867-2c1eeace9b9d\") " pod="openshift-lws-operator/lws-controller-manager-7c59ccc8d7-mgk5b" Jul 16 16:57:34.250830 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:34.250807 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e651a638-235f-4c0e-8867-2c1eeace9b9d-manager-config\") pod \"lws-controller-manager-7c59ccc8d7-mgk5b\" (UID: \"e651a638-235f-4c0e-8867-2c1eeace9b9d\") " pod="openshift-lws-operator/lws-controller-manager-7c59ccc8d7-mgk5b" Jul 16 16:57:34.252625 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:34.252592 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e651a638-235f-4c0e-8867-2c1eeace9b9d-metrics-cert\") pod \"lws-controller-manager-7c59ccc8d7-mgk5b\" (UID: \"e651a638-235f-4c0e-8867-2c1eeace9b9d\") " pod="openshift-lws-operator/lws-controller-manager-7c59ccc8d7-mgk5b" Jul 16 16:57:34.252720 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:34.252680 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e651a638-235f-4c0e-8867-2c1eeace9b9d-cert\") pod \"lws-controller-manager-7c59ccc8d7-mgk5b\" (UID: \"e651a638-235f-4c0e-8867-2c1eeace9b9d\") " pod="openshift-lws-operator/lws-controller-manager-7c59ccc8d7-mgk5b" Jul 16 16:57:34.259049 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:34.259003 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnpsr\" (UniqueName: \"kubernetes.io/projected/e651a638-235f-4c0e-8867-2c1eeace9b9d-kube-api-access-xnpsr\") pod \"lws-controller-manager-7c59ccc8d7-mgk5b\" (UID: \"e651a638-235f-4c0e-8867-2c1eeace9b9d\") " pod="openshift-lws-operator/lws-controller-manager-7c59ccc8d7-mgk5b" Jul 16 16:57:34.327900 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:34.327876 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7c59ccc8d7-mgk5b" Jul 16 16:57:34.461096 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:34.461074 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7c59ccc8d7-mgk5b"] Jul 16 16:57:34.463033 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:57:34.462993 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode651a638_235f_4c0e_8867_2c1eeace9b9d.slice/crio-9e63eccef4b73bf8398a6b5e7492f3ac0bb7d0b367153dbe8432bdf912ddfb68 WatchSource:0}: Error finding container 9e63eccef4b73bf8398a6b5e7492f3ac0bb7d0b367153dbe8432bdf912ddfb68: Status 404 returned error can't find the container with id 9e63eccef4b73bf8398a6b5e7492f3ac0bb7d0b367153dbe8432bdf912ddfb68 Jul 16 16:57:34.525377 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:34.525324 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7c59ccc8d7-mgk5b" event={"ID":"e651a638-235f-4c0e-8867-2c1eeace9b9d","Type":"ContainerStarted","Data":"9e63eccef4b73bf8398a6b5e7492f3ac0bb7d0b367153dbe8432bdf912ddfb68"} Jul 16 16:57:36.535965 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:36.535935 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7c59ccc8d7-mgk5b" event={"ID":"e651a638-235f-4c0e-8867-2c1eeace9b9d","Type":"ContainerStarted","Data":"779ef40bc11ad1487d92479d3e7723069c8a85caaf2beded067949b7651ab5a1"} Jul 16 16:57:36.536312 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:36.536058 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7c59ccc8d7-mgk5b" Jul 16 16:57:36.564988 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:36.564944 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7c59ccc8d7-mgk5b" podStartSLOduration=2.086050312 podStartE2EDuration="3.564929498s" totalImagesPullingTime="1.478879186s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:57:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:57:36.563148247 +0000 UTC m=+408.304723557" watchObservedRunningTime="2026-07-16 16:57:36.564929498 +0000 UTC m=+408.306504793" Jul 16 16:57:37.430825 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:37.430792 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-l5d8m"] Jul 16 16:57:37.433940 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:37.433924 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-l5d8m" Jul 16 16:57:37.437023 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:37.436987 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 16 16:57:37.437193 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:37.437173 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-qcb45\"" Jul 16 16:57:37.446682 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:37.446656 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-l5d8m"] Jul 16 16:57:37.576901 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:37.576877 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wt2w\" (UniqueName: \"kubernetes.io/projected/581fb7d4-c35a-47cd-9d2f-78dce3928a1a-kube-api-access-7wt2w\") pod \"model-serving-api-54d89bd5fc-l5d8m\" (UID: \"581fb7d4-c35a-47cd-9d2f-78dce3928a1a\") " pod="opendatahub/model-serving-api-54d89bd5fc-l5d8m" Jul 16 16:57:37.577209 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:37.576908 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/581fb7d4-c35a-47cd-9d2f-78dce3928a1a-tls-certs\") pod \"model-serving-api-54d89bd5fc-l5d8m\" (UID: \"581fb7d4-c35a-47cd-9d2f-78dce3928a1a\") " pod="opendatahub/model-serving-api-54d89bd5fc-l5d8m" Jul 16 16:57:37.678067 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:37.678035 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wt2w\" (UniqueName: \"kubernetes.io/projected/581fb7d4-c35a-47cd-9d2f-78dce3928a1a-kube-api-access-7wt2w\") pod \"model-serving-api-54d89bd5fc-l5d8m\" (UID: \"581fb7d4-c35a-47cd-9d2f-78dce3928a1a\") " pod="opendatahub/model-serving-api-54d89bd5fc-l5d8m" Jul 16 16:57:37.678233 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:37.678078 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/581fb7d4-c35a-47cd-9d2f-78dce3928a1a-tls-certs\") pod \"model-serving-api-54d89bd5fc-l5d8m\" (UID: \"581fb7d4-c35a-47cd-9d2f-78dce3928a1a\") " pod="opendatahub/model-serving-api-54d89bd5fc-l5d8m" Jul 16 16:57:37.678233 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:57:37.678188 2571 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 16 16:57:37.678356 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:57:37.678263 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/581fb7d4-c35a-47cd-9d2f-78dce3928a1a-tls-certs podName:581fb7d4-c35a-47cd-9d2f-78dce3928a1a nodeName:}" failed. No retries permitted until 2026-07-16 16:57:38.178240977 +0000 UTC m=+409.919816265 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/581fb7d4-c35a-47cd-9d2f-78dce3928a1a-tls-certs") pod "model-serving-api-54d89bd5fc-l5d8m" (UID: "581fb7d4-c35a-47cd-9d2f-78dce3928a1a") : secret "model-serving-api-tls" not found Jul 16 16:57:37.688248 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:37.688177 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wt2w\" (UniqueName: \"kubernetes.io/projected/581fb7d4-c35a-47cd-9d2f-78dce3928a1a-kube-api-access-7wt2w\") pod \"model-serving-api-54d89bd5fc-l5d8m\" (UID: \"581fb7d4-c35a-47cd-9d2f-78dce3928a1a\") " pod="opendatahub/model-serving-api-54d89bd5fc-l5d8m" Jul 16 16:57:38.182154 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:38.182126 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/581fb7d4-c35a-47cd-9d2f-78dce3928a1a-tls-certs\") pod \"model-serving-api-54d89bd5fc-l5d8m\" (UID: \"581fb7d4-c35a-47cd-9d2f-78dce3928a1a\") " pod="opendatahub/model-serving-api-54d89bd5fc-l5d8m" Jul 16 16:57:38.184461 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:38.184436 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/581fb7d4-c35a-47cd-9d2f-78dce3928a1a-tls-certs\") pod \"model-serving-api-54d89bd5fc-l5d8m\" (UID: \"581fb7d4-c35a-47cd-9d2f-78dce3928a1a\") " pod="opendatahub/model-serving-api-54d89bd5fc-l5d8m" Jul 16 16:57:38.343864 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:38.343837 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-l5d8m" Jul 16 16:57:38.468143 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:38.468122 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-l5d8m"] Jul 16 16:57:38.470097 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:57:38.470072 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod581fb7d4_c35a_47cd_9d2f_78dce3928a1a.slice/crio-3fb6c5f7dfde1b0263cffda52229af1e735c1e51334bd3158a8b48cbe2766352 WatchSource:0}: Error finding container 3fb6c5f7dfde1b0263cffda52229af1e735c1e51334bd3158a8b48cbe2766352: Status 404 returned error can't find the container with id 3fb6c5f7dfde1b0263cffda52229af1e735c1e51334bd3158a8b48cbe2766352 Jul 16 16:57:38.546509 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:38.546475 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-l5d8m" event={"ID":"581fb7d4-c35a-47cd-9d2f-78dce3928a1a","Type":"ContainerStarted","Data":"3fb6c5f7dfde1b0263cffda52229af1e735c1e51334bd3158a8b48cbe2766352"} Jul 16 16:57:40.558525 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:40.558492 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-l5d8m" event={"ID":"581fb7d4-c35a-47cd-9d2f-78dce3928a1a","Type":"ContainerStarted","Data":"b81fd9ff63946ece26e7705a953685a56b0224753494ae2f4747af6733b5f63a"} Jul 16 16:57:40.558935 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:40.558624 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-l5d8m" Jul 16 16:57:40.580381 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:40.580328 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-l5d8m" podStartSLOduration=2.0768120899999998 podStartE2EDuration="3.580314518s" totalImagesPullingTime="1.503502428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:57:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:57:40.578876686 +0000 UTC m=+412.320451983" watchObservedRunningTime="2026-07-16 16:57:40.580314518 +0000 UTC m=+412.321889812" Jul 16 16:57:43.639847 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:43.639814 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6"] Jul 16 16:57:43.643055 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:43.643034 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6" Jul 16 16:57:43.647326 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:43.647298 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 16 16:57:43.648488 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:43.648463 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-vbhct\"" Jul 16 16:57:43.659537 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:43.659504 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6"] Jul 16 16:57:43.723744 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:43.723687 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/610c0590-6a17-4104-bf41-92c6040cdfa6-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-z96g6\" (UID: \"610c0590-6a17-4104-bf41-92c6040cdfa6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6" Jul 16 16:57:43.723744 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:43.723723 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/610c0590-6a17-4104-bf41-92c6040cdfa6-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-z96g6\" (UID: \"610c0590-6a17-4104-bf41-92c6040cdfa6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6" Jul 16 16:57:43.723855 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:43.723794 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/610c0590-6a17-4104-bf41-92c6040cdfa6-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-z96g6\" (UID: \"610c0590-6a17-4104-bf41-92c6040cdfa6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6" Jul 16 16:57:43.723855 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:43.723813 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/610c0590-6a17-4104-bf41-92c6040cdfa6-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-z96g6\" (UID: \"610c0590-6a17-4104-bf41-92c6040cdfa6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6" Jul 16 16:57:43.723937 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:43.723881 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/610c0590-6a17-4104-bf41-92c6040cdfa6-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-z96g6\" (UID: \"610c0590-6a17-4104-bf41-92c6040cdfa6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6" Jul 16 16:57:43.723937 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:43.723922 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/610c0590-6a17-4104-bf41-92c6040cdfa6-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-z96g6\" (UID: \"610c0590-6a17-4104-bf41-92c6040cdfa6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6" Jul 16 16:57:43.724007 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:43.723940 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/610c0590-6a17-4104-bf41-92c6040cdfa6-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-z96g6\" (UID: \"610c0590-6a17-4104-bf41-92c6040cdfa6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6" Jul 16 16:57:43.724007 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:43.723965 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwjkw\" (UniqueName: \"kubernetes.io/projected/610c0590-6a17-4104-bf41-92c6040cdfa6-kube-api-access-lwjkw\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-z96g6\" (UID: \"610c0590-6a17-4104-bf41-92c6040cdfa6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6" Jul 16 16:57:43.724076 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:43.724020 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/610c0590-6a17-4104-bf41-92c6040cdfa6-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-z96g6\" (UID: \"610c0590-6a17-4104-bf41-92c6040cdfa6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6" Jul 16 16:57:43.824402 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:43.824375 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/610c0590-6a17-4104-bf41-92c6040cdfa6-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-z96g6\" (UID: \"610c0590-6a17-4104-bf41-92c6040cdfa6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6" Jul 16 16:57:43.824402 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:43.824402 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/610c0590-6a17-4104-bf41-92c6040cdfa6-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-z96g6\" (UID: \"610c0590-6a17-4104-bf41-92c6040cdfa6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6" Jul 16 16:57:43.824558 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:43.824427 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/610c0590-6a17-4104-bf41-92c6040cdfa6-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-z96g6\" (UID: \"610c0590-6a17-4104-bf41-92c6040cdfa6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6" Jul 16 16:57:43.824558 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:43.824444 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/610c0590-6a17-4104-bf41-92c6040cdfa6-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-z96g6\" (UID: \"610c0590-6a17-4104-bf41-92c6040cdfa6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6" Jul 16 16:57:43.824558 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:43.824471 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/610c0590-6a17-4104-bf41-92c6040cdfa6-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-z96g6\" (UID: \"610c0590-6a17-4104-bf41-92c6040cdfa6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6" Jul 16 16:57:43.824558 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:43.824504 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/610c0590-6a17-4104-bf41-92c6040cdfa6-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-z96g6\" (UID: \"610c0590-6a17-4104-bf41-92c6040cdfa6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6" Jul 16 16:57:43.824558 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:43.824526 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/610c0590-6a17-4104-bf41-92c6040cdfa6-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-z96g6\" (UID: \"610c0590-6a17-4104-bf41-92c6040cdfa6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6" Jul 16 16:57:43.824558 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:43.824554 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwjkw\" (UniqueName: \"kubernetes.io/projected/610c0590-6a17-4104-bf41-92c6040cdfa6-kube-api-access-lwjkw\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-z96g6\" (UID: \"610c0590-6a17-4104-bf41-92c6040cdfa6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6" Jul 16 16:57:43.824866 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:43.824588 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/610c0590-6a17-4104-bf41-92c6040cdfa6-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-z96g6\" (UID: \"610c0590-6a17-4104-bf41-92c6040cdfa6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6" Jul 16 16:57:43.824967 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:43.824945 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/610c0590-6a17-4104-bf41-92c6040cdfa6-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-z96g6\" (UID: \"610c0590-6a17-4104-bf41-92c6040cdfa6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6" Jul 16 16:57:43.825045 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:43.825009 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/610c0590-6a17-4104-bf41-92c6040cdfa6-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-z96g6\" (UID: \"610c0590-6a17-4104-bf41-92c6040cdfa6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6" Jul 16 16:57:43.825099 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:43.825070 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/610c0590-6a17-4104-bf41-92c6040cdfa6-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-z96g6\" (UID: \"610c0590-6a17-4104-bf41-92c6040cdfa6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6" Jul 16 16:57:43.825191 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:43.825163 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/610c0590-6a17-4104-bf41-92c6040cdfa6-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-z96g6\" (UID: \"610c0590-6a17-4104-bf41-92c6040cdfa6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6" Jul 16 16:57:43.825325 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:43.825303 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/610c0590-6a17-4104-bf41-92c6040cdfa6-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-z96g6\" (UID: \"610c0590-6a17-4104-bf41-92c6040cdfa6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6" Jul 16 16:57:43.827090 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:43.827066 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/610c0590-6a17-4104-bf41-92c6040cdfa6-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-z96g6\" (UID: \"610c0590-6a17-4104-bf41-92c6040cdfa6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6" Jul 16 16:57:43.827305 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:43.827287 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/610c0590-6a17-4104-bf41-92c6040cdfa6-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-z96g6\" (UID: \"610c0590-6a17-4104-bf41-92c6040cdfa6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6" Jul 16 16:57:43.835952 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:43.835924 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/610c0590-6a17-4104-bf41-92c6040cdfa6-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-z96g6\" (UID: \"610c0590-6a17-4104-bf41-92c6040cdfa6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6" Jul 16 16:57:43.839574 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:43.839555 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwjkw\" (UniqueName: \"kubernetes.io/projected/610c0590-6a17-4104-bf41-92c6040cdfa6-kube-api-access-lwjkw\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-z96g6\" (UID: \"610c0590-6a17-4104-bf41-92c6040cdfa6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6" Jul 16 16:57:43.953242 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:43.953218 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6" Jul 16 16:57:44.102585 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:44.102555 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6"] Jul 16 16:57:44.103661 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:57:44.103632 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod610c0590_6a17_4104_bf41_92c6040cdfa6.slice/crio-863fc840d88c9a4033c6f6bd93c42365bd0160ee1c5190c83884371c68920341 WatchSource:0}: Error finding container 863fc840d88c9a4033c6f6bd93c42365bd0160ee1c5190c83884371c68920341: Status 404 returned error can't find the container with id 863fc840d88c9a4033c6f6bd93c42365bd0160ee1c5190c83884371c68920341 Jul 16 16:57:44.578992 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:44.578955 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6" event={"ID":"610c0590-6a17-4104-bf41-92c6040cdfa6","Type":"ContainerStarted","Data":"863fc840d88c9a4033c6f6bd93c42365bd0160ee1c5190c83884371c68920341"} Jul 16 16:57:47.542946 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:47.542915 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7c59ccc8d7-mgk5b" Jul 16 16:57:48.554433 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:48.554397 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6"] Jul 16 16:57:48.787986 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:48.787942 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 16:57:48.788085 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:48.788030 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 16:57:48.788085 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:48.788069 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 16:57:48.912975 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:48.912942 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-7zf2x"] Jul 16 16:57:48.916279 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:48.916260 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7zf2x" Jul 16 16:57:48.920301 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:48.920283 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-znhs6\"" Jul 16 16:57:48.920483 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:48.920461 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 16 16:57:48.930944 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:48.930916 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-85csp"] Jul 16 16:57:48.934566 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:48.934447 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-7zf2x"] Jul 16 16:57:48.934566 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:48.934551 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-85csp" Jul 16 16:57:48.937992 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:48.937953 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 16 16:57:48.940540 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:48.940104 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-nnmfc\"" Jul 16 16:57:48.953550 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:48.953519 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-85csp"] Jul 16 16:57:49.060699 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:49.060640 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57n7p\" (UniqueName: \"kubernetes.io/projected/28df4feb-0098-4de5-82c3-d983879f0f41-kube-api-access-57n7p\") pod \"llmisvc-controller-manager-777c6bd7d5-85csp\" (UID: \"28df4feb-0098-4de5-82c3-d983879f0f41\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-85csp" Jul 16 16:57:49.060699 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:49.060682 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/28df4feb-0098-4de5-82c3-d983879f0f41-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-85csp\" (UID: \"28df4feb-0098-4de5-82c3-d983879f0f41\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-85csp" Jul 16 16:57:49.060839 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:49.060800 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b162a8ad-e02f-4bb7-969a-b27787efcf7b-cert\") pod \"kserve-controller-manager-d6f4b65b8-7zf2x\" (UID: \"b162a8ad-e02f-4bb7-969a-b27787efcf7b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7zf2x" Jul 16 16:57:49.060875 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:49.060839 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jp5ld\" (UniqueName: \"kubernetes.io/projected/b162a8ad-e02f-4bb7-969a-b27787efcf7b-kube-api-access-jp5ld\") pod \"kserve-controller-manager-d6f4b65b8-7zf2x\" (UID: \"b162a8ad-e02f-4bb7-969a-b27787efcf7b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7zf2x" Jul 16 16:57:49.161906 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:49.161883 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b162a8ad-e02f-4bb7-969a-b27787efcf7b-cert\") pod \"kserve-controller-manager-d6f4b65b8-7zf2x\" (UID: \"b162a8ad-e02f-4bb7-969a-b27787efcf7b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7zf2x" Jul 16 16:57:49.162007 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:49.161915 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jp5ld\" (UniqueName: \"kubernetes.io/projected/b162a8ad-e02f-4bb7-969a-b27787efcf7b-kube-api-access-jp5ld\") pod \"kserve-controller-manager-d6f4b65b8-7zf2x\" (UID: \"b162a8ad-e02f-4bb7-969a-b27787efcf7b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7zf2x" Jul 16 16:57:49.162007 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:49.161944 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57n7p\" (UniqueName: \"kubernetes.io/projected/28df4feb-0098-4de5-82c3-d983879f0f41-kube-api-access-57n7p\") pod \"llmisvc-controller-manager-777c6bd7d5-85csp\" (UID: \"28df4feb-0098-4de5-82c3-d983879f0f41\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-85csp" Jul 16 16:57:49.162007 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:49.161994 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/28df4feb-0098-4de5-82c3-d983879f0f41-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-85csp\" (UID: \"28df4feb-0098-4de5-82c3-d983879f0f41\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-85csp" Jul 16 16:57:49.164791 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:49.164767 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/28df4feb-0098-4de5-82c3-d983879f0f41-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-85csp\" (UID: \"28df4feb-0098-4de5-82c3-d983879f0f41\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-85csp" Jul 16 16:57:49.164931 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:49.164913 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b162a8ad-e02f-4bb7-969a-b27787efcf7b-cert\") pod \"kserve-controller-manager-d6f4b65b8-7zf2x\" (UID: \"b162a8ad-e02f-4bb7-969a-b27787efcf7b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7zf2x" Jul 16 16:57:49.175471 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:49.175441 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jp5ld\" (UniqueName: \"kubernetes.io/projected/b162a8ad-e02f-4bb7-969a-b27787efcf7b-kube-api-access-jp5ld\") pod \"kserve-controller-manager-d6f4b65b8-7zf2x\" (UID: \"b162a8ad-e02f-4bb7-969a-b27787efcf7b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7zf2x" Jul 16 16:57:49.177970 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:49.177952 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57n7p\" (UniqueName: \"kubernetes.io/projected/28df4feb-0098-4de5-82c3-d983879f0f41-kube-api-access-57n7p\") pod \"llmisvc-controller-manager-777c6bd7d5-85csp\" (UID: \"28df4feb-0098-4de5-82c3-d983879f0f41\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-85csp" Jul 16 16:57:49.230064 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:49.230043 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7zf2x" Jul 16 16:57:49.243773 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:49.243739 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-85csp" Jul 16 16:57:49.416097 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:49.416068 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-7zf2x"] Jul 16 16:57:49.418567 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:57:49.418544 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb162a8ad_e02f_4bb7_969a_b27787efcf7b.slice/crio-d360eae98d9fc00de0b7cbe3653b4f385c6598b6685a13ce5a998539e04c3cac WatchSource:0}: Error finding container d360eae98d9fc00de0b7cbe3653b4f385c6598b6685a13ce5a998539e04c3cac: Status 404 returned error can't find the container with id d360eae98d9fc00de0b7cbe3653b4f385c6598b6685a13ce5a998539e04c3cac Jul 16 16:57:49.431316 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:49.431294 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-85csp"] Jul 16 16:57:49.431809 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:57:49.431787 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod28df4feb_0098_4de5_82c3_d983879f0f41.slice/crio-cb7c860f08cc919ebba23c2464bb673c28199e567b5aa01a6567cac61ee23cbf WatchSource:0}: Error finding container cb7c860f08cc919ebba23c2464bb673c28199e567b5aa01a6567cac61ee23cbf: Status 404 returned error can't find the container with id cb7c860f08cc919ebba23c2464bb673c28199e567b5aa01a6567cac61ee23cbf Jul 16 16:57:49.611890 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:49.611814 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-85csp" event={"ID":"28df4feb-0098-4de5-82c3-d983879f0f41","Type":"ContainerStarted","Data":"cb7c860f08cc919ebba23c2464bb673c28199e567b5aa01a6567cac61ee23cbf"} Jul 16 16:57:49.613288 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:49.613264 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6" event={"ID":"610c0590-6a17-4104-bf41-92c6040cdfa6","Type":"ContainerStarted","Data":"42deacd74d9a54e4c72a0d4d04238f166221dbbf13fd8158381193224230d411"} Jul 16 16:57:49.613392 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:49.613353 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6" podUID="610c0590-6a17-4104-bf41-92c6040cdfa6" containerName="istio-proxy" containerID="cri-o://42deacd74d9a54e4c72a0d4d04238f166221dbbf13fd8158381193224230d411" gracePeriod=30 Jul 16 16:57:49.614511 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:49.614486 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7zf2x" event={"ID":"b162a8ad-e02f-4bb7-969a-b27787efcf7b","Type":"ContainerStarted","Data":"d360eae98d9fc00de0b7cbe3653b4f385c6598b6685a13ce5a998539e04c3cac"} Jul 16 16:57:49.647216 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:49.647174 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6" podStartSLOduration=1.965125617 podStartE2EDuration="6.647160636s" totalImagesPullingTime="4.682035019s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:57:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:57:49.645286121 +0000 UTC m=+421.386861419" watchObservedRunningTime="2026-07-16 16:57:49.647160636 +0000 UTC m=+421.388735931" Jul 16 16:57:51.570096 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:51.569951 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-l5d8m" Jul 16 16:57:53.640182 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:53.640138 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7zf2x" event={"ID":"b162a8ad-e02f-4bb7-969a-b27787efcf7b","Type":"ContainerStarted","Data":"3e10fc2e14b0c2b25e9e59db11b3945e31e695acdbea2441f61905ce586f9694"} Jul 16 16:57:53.640606 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:53.640274 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7zf2x" Jul 16 16:57:53.641641 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:53.641617 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-85csp" event={"ID":"28df4feb-0098-4de5-82c3-d983879f0f41","Type":"ContainerStarted","Data":"e963e772a3ba7ac8efa49c45038d62e6f0315ae4a5361ae5b2d7d55b201c2024"} Jul 16 16:57:53.641770 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:53.641743 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-85csp" Jul 16 16:57:53.665017 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:53.664970 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7zf2x" podStartSLOduration=2.419859007 podStartE2EDuration="5.66495846s" totalImagesPullingTime="3.245099453s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:57:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:57:53.664364953 +0000 UTC m=+425.405940248" watchObservedRunningTime="2026-07-16 16:57:53.66495846 +0000 UTC m=+425.406533755" Jul 16 16:57:53.687116 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:53.687076 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-85csp" podStartSLOduration=2.45127441 podStartE2EDuration="5.687063448s" totalImagesPullingTime="3.235789038s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:57:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:57:53.685049188 +0000 UTC m=+425.426624484" watchObservedRunningTime="2026-07-16 16:57:53.687063448 +0000 UTC m=+425.428638744" Jul 16 16:57:54.858191 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:54.858170 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6" Jul 16 16:57:55.011048 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:55.011022 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/610c0590-6a17-4104-bf41-92c6040cdfa6-istio-envoy\") pod \"610c0590-6a17-4104-bf41-92c6040cdfa6\" (UID: \"610c0590-6a17-4104-bf41-92c6040cdfa6\") " Jul 16 16:57:55.011214 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:55.011076 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/610c0590-6a17-4104-bf41-92c6040cdfa6-workload-socket\") pod \"610c0590-6a17-4104-bf41-92c6040cdfa6\" (UID: \"610c0590-6a17-4104-bf41-92c6040cdfa6\") " Jul 16 16:57:55.011214 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:55.011101 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/610c0590-6a17-4104-bf41-92c6040cdfa6-credential-socket\") pod \"610c0590-6a17-4104-bf41-92c6040cdfa6\" (UID: \"610c0590-6a17-4104-bf41-92c6040cdfa6\") " Jul 16 16:57:55.011214 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:55.011124 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/610c0590-6a17-4104-bf41-92c6040cdfa6-istio-token\") pod \"610c0590-6a17-4104-bf41-92c6040cdfa6\" (UID: \"610c0590-6a17-4104-bf41-92c6040cdfa6\") " Jul 16 16:57:55.011214 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:55.011151 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/610c0590-6a17-4104-bf41-92c6040cdfa6-workload-certs\") pod \"610c0590-6a17-4104-bf41-92c6040cdfa6\" (UID: \"610c0590-6a17-4104-bf41-92c6040cdfa6\") " Jul 16 16:57:55.011214 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:55.011178 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/610c0590-6a17-4104-bf41-92c6040cdfa6-istio-podinfo\") pod \"610c0590-6a17-4104-bf41-92c6040cdfa6\" (UID: \"610c0590-6a17-4104-bf41-92c6040cdfa6\") " Jul 16 16:57:55.011214 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:55.011202 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/610c0590-6a17-4104-bf41-92c6040cdfa6-istio-data\") pod \"610c0590-6a17-4104-bf41-92c6040cdfa6\" (UID: \"610c0590-6a17-4104-bf41-92c6040cdfa6\") " Jul 16 16:57:55.011494 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:55.011240 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/610c0590-6a17-4104-bf41-92c6040cdfa6-istiod-ca-cert\") pod \"610c0590-6a17-4104-bf41-92c6040cdfa6\" (UID: \"610c0590-6a17-4104-bf41-92c6040cdfa6\") " Jul 16 16:57:55.011494 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:55.011273 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lwjkw\" (UniqueName: \"kubernetes.io/projected/610c0590-6a17-4104-bf41-92c6040cdfa6-kube-api-access-lwjkw\") pod \"610c0590-6a17-4104-bf41-92c6040cdfa6\" (UID: \"610c0590-6a17-4104-bf41-92c6040cdfa6\") " Jul 16 16:57:55.011494 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:55.011350 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/610c0590-6a17-4104-bf41-92c6040cdfa6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "610c0590-6a17-4104-bf41-92c6040cdfa6" (UID: "610c0590-6a17-4104-bf41-92c6040cdfa6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:57:55.011494 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:55.011465 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/610c0590-6a17-4104-bf41-92c6040cdfa6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "610c0590-6a17-4104-bf41-92c6040cdfa6" (UID: "610c0590-6a17-4104-bf41-92c6040cdfa6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:57:55.011494 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:55.011477 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/610c0590-6a17-4104-bf41-92c6040cdfa6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "610c0590-6a17-4104-bf41-92c6040cdfa6" (UID: "610c0590-6a17-4104-bf41-92c6040cdfa6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:57:55.011720 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:55.011646 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/610c0590-6a17-4104-bf41-92c6040cdfa6-workload-socket\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:57:55.011720 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:55.011665 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/610c0590-6a17-4104-bf41-92c6040cdfa6-credential-socket\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:57:55.011720 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:55.011680 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/610c0590-6a17-4104-bf41-92c6040cdfa6-workload-certs\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:57:55.011720 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:55.011660 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/610c0590-6a17-4104-bf41-92c6040cdfa6-istio-data" (OuterVolumeSpecName: "istio-data") pod "610c0590-6a17-4104-bf41-92c6040cdfa6" (UID: "610c0590-6a17-4104-bf41-92c6040cdfa6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:57:55.011904 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:55.011778 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/610c0590-6a17-4104-bf41-92c6040cdfa6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "610c0590-6a17-4104-bf41-92c6040cdfa6" (UID: "610c0590-6a17-4104-bf41-92c6040cdfa6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 16:57:55.013345 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:55.013314 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/610c0590-6a17-4104-bf41-92c6040cdfa6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "610c0590-6a17-4104-bf41-92c6040cdfa6" (UID: "610c0590-6a17-4104-bf41-92c6040cdfa6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 16:57:55.013464 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:55.013346 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/610c0590-6a17-4104-bf41-92c6040cdfa6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "610c0590-6a17-4104-bf41-92c6040cdfa6" (UID: "610c0590-6a17-4104-bf41-92c6040cdfa6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:57:55.013678 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:55.013661 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/610c0590-6a17-4104-bf41-92c6040cdfa6-istio-token" (OuterVolumeSpecName: "istio-token") pod "610c0590-6a17-4104-bf41-92c6040cdfa6" (UID: "610c0590-6a17-4104-bf41-92c6040cdfa6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:57:55.013753 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:55.013670 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/610c0590-6a17-4104-bf41-92c6040cdfa6-kube-api-access-lwjkw" (OuterVolumeSpecName: "kube-api-access-lwjkw") pod "610c0590-6a17-4104-bf41-92c6040cdfa6" (UID: "610c0590-6a17-4104-bf41-92c6040cdfa6"). InnerVolumeSpecName "kube-api-access-lwjkw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:57:55.112766 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:55.112718 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/610c0590-6a17-4104-bf41-92c6040cdfa6-istiod-ca-cert\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:57:55.112766 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:55.112761 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwjkw\" (UniqueName: \"kubernetes.io/projected/610c0590-6a17-4104-bf41-92c6040cdfa6-kube-api-access-lwjkw\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:57:55.112905 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:55.112771 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/610c0590-6a17-4104-bf41-92c6040cdfa6-istio-envoy\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:57:55.112905 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:55.112780 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/610c0590-6a17-4104-bf41-92c6040cdfa6-istio-token\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:57:55.112905 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:55.112787 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/610c0590-6a17-4104-bf41-92c6040cdfa6-istio-podinfo\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:57:55.112905 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:55.112794 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/610c0590-6a17-4104-bf41-92c6040cdfa6-istio-data\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:57:55.655979 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:55.655950 2571 generic.go:356] "Generic (PLEG): container finished" podID="610c0590-6a17-4104-bf41-92c6040cdfa6" containerID="42deacd74d9a54e4c72a0d4d04238f166221dbbf13fd8158381193224230d411" exitCode=0 Jul 16 16:57:55.656139 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:55.656001 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6" Jul 16 16:57:55.656139 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:55.656033 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6" event={"ID":"610c0590-6a17-4104-bf41-92c6040cdfa6","Type":"ContainerDied","Data":"42deacd74d9a54e4c72a0d4d04238f166221dbbf13fd8158381193224230d411"} Jul 16 16:57:55.656139 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:55.656097 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6" event={"ID":"610c0590-6a17-4104-bf41-92c6040cdfa6","Type":"ContainerDied","Data":"863fc840d88c9a4033c6f6bd93c42365bd0160ee1c5190c83884371c68920341"} Jul 16 16:57:55.656139 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:55.656124 2571 scope.go:117] "RemoveContainer" containerID="42deacd74d9a54e4c72a0d4d04238f166221dbbf13fd8158381193224230d411" Jul 16 16:57:55.666233 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:55.666212 2571 scope.go:117] "RemoveContainer" containerID="42deacd74d9a54e4c72a0d4d04238f166221dbbf13fd8158381193224230d411" Jul 16 16:57:55.666539 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:57:55.666513 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42deacd74d9a54e4c72a0d4d04238f166221dbbf13fd8158381193224230d411\": container with ID starting with 42deacd74d9a54e4c72a0d4d04238f166221dbbf13fd8158381193224230d411 not found: ID does not exist" containerID="42deacd74d9a54e4c72a0d4d04238f166221dbbf13fd8158381193224230d411" Jul 16 16:57:55.666607 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:55.666548 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42deacd74d9a54e4c72a0d4d04238f166221dbbf13fd8158381193224230d411"} err="failed to get container status \"42deacd74d9a54e4c72a0d4d04238f166221dbbf13fd8158381193224230d411\": rpc error: code = NotFound desc = could not find container \"42deacd74d9a54e4c72a0d4d04238f166221dbbf13fd8158381193224230d411\": container with ID starting with 42deacd74d9a54e4c72a0d4d04238f166221dbbf13fd8158381193224230d411 not found: ID does not exist" Jul 16 16:57:55.681204 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:55.681182 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6"] Jul 16 16:57:55.687946 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:55.687923 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-z96g6"] Jul 16 16:57:56.776664 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:57:56.776632 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="610c0590-6a17-4104-bf41-92c6040cdfa6" path="/var/lib/kubelet/pods/610c0590-6a17-4104-bf41-92c6040cdfa6/volumes" Jul 16 16:58:24.648519 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:24.648445 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-85csp" Jul 16 16:58:24.651587 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:24.651566 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7zf2x" Jul 16 16:58:52.940593 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:52.940555 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09psp4"] Jul 16 16:58:52.941026 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:52.940898 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="610c0590-6a17-4104-bf41-92c6040cdfa6" containerName="istio-proxy" Jul 16 16:58:52.941026 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:52.940911 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="610c0590-6a17-4104-bf41-92c6040cdfa6" containerName="istio-proxy" Jul 16 16:58:52.941026 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:52.940965 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="610c0590-6a17-4104-bf41-92c6040cdfa6" containerName="istio-proxy" Jul 16 16:58:52.945393 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:52.945369 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09psp4" Jul 16 16:58:52.948284 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:52.948264 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-chkj9\"" Jul 16 16:58:52.948397 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:52.948322 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 16 16:58:52.948397 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:52.948330 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 16 16:58:52.953498 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:52.953470 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09psp4"] Jul 16 16:58:53.048785 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.048750 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/04adff86-9c0b-495e-b73c-29da5b579fda-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09psp4\" (UID: \"04adff86-9c0b-495e-b73c-29da5b579fda\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09psp4" Jul 16 16:58:53.048785 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.048786 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpv78\" (UniqueName: \"kubernetes.io/projected/04adff86-9c0b-495e-b73c-29da5b579fda-kube-api-access-gpv78\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09psp4\" (UID: \"04adff86-9c0b-495e-b73c-29da5b579fda\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09psp4" Jul 16 16:58:53.048967 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.048844 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/04adff86-9c0b-495e-b73c-29da5b579fda-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09psp4\" (UID: \"04adff86-9c0b-495e-b73c-29da5b579fda\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09psp4" Jul 16 16:58:53.149247 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.149214 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/04adff86-9c0b-495e-b73c-29da5b579fda-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09psp4\" (UID: \"04adff86-9c0b-495e-b73c-29da5b579fda\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09psp4" Jul 16 16:58:53.149379 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.149252 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpv78\" (UniqueName: \"kubernetes.io/projected/04adff86-9c0b-495e-b73c-29da5b579fda-kube-api-access-gpv78\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09psp4\" (UID: \"04adff86-9c0b-495e-b73c-29da5b579fda\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09psp4" Jul 16 16:58:53.149379 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.149279 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/04adff86-9c0b-495e-b73c-29da5b579fda-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09psp4\" (UID: \"04adff86-9c0b-495e-b73c-29da5b579fda\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09psp4" Jul 16 16:58:53.149574 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.149556 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/04adff86-9c0b-495e-b73c-29da5b579fda-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09psp4\" (UID: \"04adff86-9c0b-495e-b73c-29da5b579fda\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09psp4" Jul 16 16:58:53.149616 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.149587 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/04adff86-9c0b-495e-b73c-29da5b579fda-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09psp4\" (UID: \"04adff86-9c0b-495e-b73c-29da5b579fda\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09psp4" Jul 16 16:58:53.158869 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.158845 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpv78\" (UniqueName: \"kubernetes.io/projected/04adff86-9c0b-495e-b73c-29da5b579fda-kube-api-access-gpv78\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09psp4\" (UID: \"04adff86-9c0b-495e-b73c-29da5b579fda\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09psp4" Jul 16 16:58:53.255398 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.255344 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09psp4" Jul 16 16:58:53.341751 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.341709 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75998hh8"] Jul 16 16:58:53.347554 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.347534 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75998hh8" Jul 16 16:58:53.360082 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.360032 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75998hh8"] Jul 16 16:58:53.410762 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.410718 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09psp4"] Jul 16 16:58:53.413440 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:58:53.413415 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04adff86_9c0b_495e_b73c_29da5b579fda.slice/crio-2affd82acffa5243a1eae4baca8ff97350f8f915f8f030d4c34a102bad51df7e WatchSource:0}: Error finding container 2affd82acffa5243a1eae4baca8ff97350f8f915f8f030d4c34a102bad51df7e: Status 404 returned error can't find the container with id 2affd82acffa5243a1eae4baca8ff97350f8f915f8f030d4c34a102bad51df7e Jul 16 16:58:53.451576 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.451552 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ed08f90-c844-43f1-9c65-ab8bede0ee73-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75998hh8\" (UID: \"4ed08f90-c844-43f1-9c65-ab8bede0ee73\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75998hh8" Jul 16 16:58:53.451675 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.451588 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fm7t5\" (UniqueName: \"kubernetes.io/projected/4ed08f90-c844-43f1-9c65-ab8bede0ee73-kube-api-access-fm7t5\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75998hh8\" (UID: \"4ed08f90-c844-43f1-9c65-ab8bede0ee73\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75998hh8" Jul 16 16:58:53.451675 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.451613 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ed08f90-c844-43f1-9c65-ab8bede0ee73-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75998hh8\" (UID: \"4ed08f90-c844-43f1-9c65-ab8bede0ee73\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75998hh8" Jul 16 16:58:53.552500 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.552474 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ed08f90-c844-43f1-9c65-ab8bede0ee73-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75998hh8\" (UID: \"4ed08f90-c844-43f1-9c65-ab8bede0ee73\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75998hh8" Jul 16 16:58:53.552667 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.552645 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ed08f90-c844-43f1-9c65-ab8bede0ee73-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75998hh8\" (UID: \"4ed08f90-c844-43f1-9c65-ab8bede0ee73\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75998hh8" Jul 16 16:58:53.552756 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.552696 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fm7t5\" (UniqueName: \"kubernetes.io/projected/4ed08f90-c844-43f1-9c65-ab8bede0ee73-kube-api-access-fm7t5\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75998hh8\" (UID: \"4ed08f90-c844-43f1-9c65-ab8bede0ee73\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75998hh8" Jul 16 16:58:53.552901 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.552882 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ed08f90-c844-43f1-9c65-ab8bede0ee73-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75998hh8\" (UID: \"4ed08f90-c844-43f1-9c65-ab8bede0ee73\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75998hh8" Jul 16 16:58:53.552959 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.552931 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ed08f90-c844-43f1-9c65-ab8bede0ee73-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75998hh8\" (UID: \"4ed08f90-c844-43f1-9c65-ab8bede0ee73\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75998hh8" Jul 16 16:58:53.562740 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.562712 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fm7t5\" (UniqueName: \"kubernetes.io/projected/4ed08f90-c844-43f1-9c65-ab8bede0ee73-kube-api-access-fm7t5\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75998hh8\" (UID: \"4ed08f90-c844-43f1-9c65-ab8bede0ee73\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75998hh8" Jul 16 16:58:53.661720 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.661690 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75998hh8" Jul 16 16:58:53.749246 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.749219 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pfl45"] Jul 16 16:58:53.753020 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.752993 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pfl45" Jul 16 16:58:53.764659 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.764631 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pfl45"] Jul 16 16:58:53.796969 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.796940 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75998hh8"] Jul 16 16:58:53.798570 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:58:53.798548 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ed08f90_c844_43f1_9c65_ab8bede0ee73.slice/crio-faab75854df93b1518167ba1ceb4f2f34898f9bcdacc6884f904d5ac1efd4048 WatchSource:0}: Error finding container faab75854df93b1518167ba1ceb4f2f34898f9bcdacc6884f904d5ac1efd4048: Status 404 returned error can't find the container with id faab75854df93b1518167ba1ceb4f2f34898f9bcdacc6884f904d5ac1efd4048 Jul 16 16:58:53.855525 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.855502 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8df978d6-8d77-40cf-a714-af26edb260f8-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pfl45\" (UID: \"8df978d6-8d77-40cf-a714-af26edb260f8\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pfl45" Jul 16 16:58:53.855649 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.855558 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8df978d6-8d77-40cf-a714-af26edb260f8-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pfl45\" (UID: \"8df978d6-8d77-40cf-a714-af26edb260f8\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pfl45" Jul 16 16:58:53.855649 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.855630 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9957t\" (UniqueName: \"kubernetes.io/projected/8df978d6-8d77-40cf-a714-af26edb260f8-kube-api-access-9957t\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pfl45\" (UID: \"8df978d6-8d77-40cf-a714-af26edb260f8\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pfl45" Jul 16 16:58:53.920665 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.920634 2571 generic.go:356] "Generic (PLEG): container finished" podID="04adff86-9c0b-495e-b73c-29da5b579fda" containerID="ace46c0300d08222712b79702f60f70667dc417115938e1eae6a568df797a6a3" exitCode=0 Jul 16 16:58:53.920871 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.920713 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09psp4" event={"ID":"04adff86-9c0b-495e-b73c-29da5b579fda","Type":"ContainerDied","Data":"ace46c0300d08222712b79702f60f70667dc417115938e1eae6a568df797a6a3"} Jul 16 16:58:53.920871 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.920777 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09psp4" event={"ID":"04adff86-9c0b-495e-b73c-29da5b579fda","Type":"ContainerStarted","Data":"2affd82acffa5243a1eae4baca8ff97350f8f915f8f030d4c34a102bad51df7e"} Jul 16 16:58:53.922309 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.922288 2571 generic.go:356] "Generic (PLEG): container finished" podID="4ed08f90-c844-43f1-9c65-ab8bede0ee73" containerID="81d2d9c5ad8b83af5d58d1291f7a986b6fcaa3fefb7abb9a10d1a2217e1d2362" exitCode=0 Jul 16 16:58:53.922433 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.922345 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75998hh8" event={"ID":"4ed08f90-c844-43f1-9c65-ab8bede0ee73","Type":"ContainerDied","Data":"81d2d9c5ad8b83af5d58d1291f7a986b6fcaa3fefb7abb9a10d1a2217e1d2362"} Jul 16 16:58:53.922433 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.922388 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75998hh8" event={"ID":"4ed08f90-c844-43f1-9c65-ab8bede0ee73","Type":"ContainerStarted","Data":"faab75854df93b1518167ba1ceb4f2f34898f9bcdacc6884f904d5ac1efd4048"} Jul 16 16:58:53.956781 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.956750 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8df978d6-8d77-40cf-a714-af26edb260f8-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pfl45\" (UID: \"8df978d6-8d77-40cf-a714-af26edb260f8\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pfl45" Jul 16 16:58:53.957258 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.956797 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8df978d6-8d77-40cf-a714-af26edb260f8-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pfl45\" (UID: \"8df978d6-8d77-40cf-a714-af26edb260f8\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pfl45" Jul 16 16:58:53.957258 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.956830 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9957t\" (UniqueName: \"kubernetes.io/projected/8df978d6-8d77-40cf-a714-af26edb260f8-kube-api-access-9957t\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pfl45\" (UID: \"8df978d6-8d77-40cf-a714-af26edb260f8\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pfl45" Jul 16 16:58:53.957258 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.957113 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8df978d6-8d77-40cf-a714-af26edb260f8-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pfl45\" (UID: \"8df978d6-8d77-40cf-a714-af26edb260f8\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pfl45" Jul 16 16:58:53.957258 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.957209 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8df978d6-8d77-40cf-a714-af26edb260f8-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pfl45\" (UID: \"8df978d6-8d77-40cf-a714-af26edb260f8\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pfl45" Jul 16 16:58:53.968344 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:53.968317 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9957t\" (UniqueName: \"kubernetes.io/projected/8df978d6-8d77-40cf-a714-af26edb260f8-kube-api-access-9957t\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pfl45\" (UID: \"8df978d6-8d77-40cf-a714-af26edb260f8\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pfl45" Jul 16 16:58:54.064465 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:54.064391 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pfl45" Jul 16 16:58:54.164558 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:54.164525 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rk7h4"] Jul 16 16:58:54.169201 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:54.169174 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rk7h4" Jul 16 16:58:54.177343 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:54.177312 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rk7h4"] Jul 16 16:58:54.192613 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:54.192592 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pfl45"] Jul 16 16:58:54.194430 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:58:54.194407 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8df978d6_8d77_40cf_a714_af26edb260f8.slice/crio-61c4f81d10a864966c9dce075562fa6b98f176644356c5cf268bdea33e34cd4c WatchSource:0}: Error finding container 61c4f81d10a864966c9dce075562fa6b98f176644356c5cf268bdea33e34cd4c: Status 404 returned error can't find the container with id 61c4f81d10a864966c9dce075562fa6b98f176644356c5cf268bdea33e34cd4c Jul 16 16:58:54.259540 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:54.259518 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9b19b46-70a5-46be-b697-dcd420eed74c-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rk7h4\" (UID: \"a9b19b46-70a5-46be-b697-dcd420eed74c\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rk7h4" Jul 16 16:58:54.259665 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:54.259592 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmnhx\" (UniqueName: \"kubernetes.io/projected/a9b19b46-70a5-46be-b697-dcd420eed74c-kube-api-access-lmnhx\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rk7h4\" (UID: \"a9b19b46-70a5-46be-b697-dcd420eed74c\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rk7h4" Jul 16 16:58:54.259665 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:54.259638 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9b19b46-70a5-46be-b697-dcd420eed74c-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rk7h4\" (UID: \"a9b19b46-70a5-46be-b697-dcd420eed74c\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rk7h4" Jul 16 16:58:54.360473 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:54.360409 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9b19b46-70a5-46be-b697-dcd420eed74c-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rk7h4\" (UID: \"a9b19b46-70a5-46be-b697-dcd420eed74c\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rk7h4" Jul 16 16:58:54.360473 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:54.360470 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmnhx\" (UniqueName: \"kubernetes.io/projected/a9b19b46-70a5-46be-b697-dcd420eed74c-kube-api-access-lmnhx\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rk7h4\" (UID: \"a9b19b46-70a5-46be-b697-dcd420eed74c\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rk7h4" Jul 16 16:58:54.360662 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:54.360496 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9b19b46-70a5-46be-b697-dcd420eed74c-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rk7h4\" (UID: \"a9b19b46-70a5-46be-b697-dcd420eed74c\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rk7h4" Jul 16 16:58:54.360833 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:54.360814 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9b19b46-70a5-46be-b697-dcd420eed74c-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rk7h4\" (UID: \"a9b19b46-70a5-46be-b697-dcd420eed74c\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rk7h4" Jul 16 16:58:54.360897 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:54.360839 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9b19b46-70a5-46be-b697-dcd420eed74c-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rk7h4\" (UID: \"a9b19b46-70a5-46be-b697-dcd420eed74c\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rk7h4" Jul 16 16:58:54.370545 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:54.370524 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmnhx\" (UniqueName: \"kubernetes.io/projected/a9b19b46-70a5-46be-b697-dcd420eed74c-kube-api-access-lmnhx\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rk7h4\" (UID: \"a9b19b46-70a5-46be-b697-dcd420eed74c\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rk7h4" Jul 16 16:58:54.483472 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:54.483446 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rk7h4" Jul 16 16:58:54.622270 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:54.622227 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rk7h4"] Jul 16 16:58:54.624188 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:58:54.624161 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9b19b46_70a5_46be_b697_dcd420eed74c.slice/crio-8fd97caf4ec2b7f7d92ed2db1fb5bccb5584835640a47658532625373110da51 WatchSource:0}: Error finding container 8fd97caf4ec2b7f7d92ed2db1fb5bccb5584835640a47658532625373110da51: Status 404 returned error can't find the container with id 8fd97caf4ec2b7f7d92ed2db1fb5bccb5584835640a47658532625373110da51 Jul 16 16:58:54.927695 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:54.927672 2571 generic.go:356] "Generic (PLEG): container finished" podID="a9b19b46-70a5-46be-b697-dcd420eed74c" containerID="b7d9c0bd90cd555e3f7fb4355af9b3a23385381bc971b29578a98d3e9debff88" exitCode=0 Jul 16 16:58:54.927834 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:54.927767 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rk7h4" event={"ID":"a9b19b46-70a5-46be-b697-dcd420eed74c","Type":"ContainerDied","Data":"b7d9c0bd90cd555e3f7fb4355af9b3a23385381bc971b29578a98d3e9debff88"} Jul 16 16:58:54.927834 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:54.927818 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rk7h4" event={"ID":"a9b19b46-70a5-46be-b697-dcd420eed74c","Type":"ContainerStarted","Data":"8fd97caf4ec2b7f7d92ed2db1fb5bccb5584835640a47658532625373110da51"} Jul 16 16:58:54.929253 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:54.929232 2571 generic.go:356] "Generic (PLEG): container finished" podID="8df978d6-8d77-40cf-a714-af26edb260f8" containerID="8ea14a9f3287ee89ff6829f5840a2bb2dd51b939223ae3f5349fed57e94b1426" exitCode=0 Jul 16 16:58:54.929339 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:54.929315 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pfl45" event={"ID":"8df978d6-8d77-40cf-a714-af26edb260f8","Type":"ContainerDied","Data":"8ea14a9f3287ee89ff6829f5840a2bb2dd51b939223ae3f5349fed57e94b1426"} Jul 16 16:58:54.929391 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:54.929359 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pfl45" event={"ID":"8df978d6-8d77-40cf-a714-af26edb260f8","Type":"ContainerStarted","Data":"61c4f81d10a864966c9dce075562fa6b98f176644356c5cf268bdea33e34cd4c"} Jul 16 16:58:54.931475 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:54.931452 2571 generic.go:356] "Generic (PLEG): container finished" podID="04adff86-9c0b-495e-b73c-29da5b579fda" containerID="652ad9f5a531b9143e9549a8fa81010cdb836127f2bd0aa81e7f00b9765711f6" exitCode=0 Jul 16 16:58:54.931574 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:54.931499 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09psp4" event={"ID":"04adff86-9c0b-495e-b73c-29da5b579fda","Type":"ContainerDied","Data":"652ad9f5a531b9143e9549a8fa81010cdb836127f2bd0aa81e7f00b9765711f6"} Jul 16 16:58:55.939389 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:55.939358 2571 generic.go:356] "Generic (PLEG): container finished" podID="4ed08f90-c844-43f1-9c65-ab8bede0ee73" containerID="4be9e7863c464a5ad71a9bd2b459fa4f3cbd7f437f16c0f9befc5ac3b93a4656" exitCode=0 Jul 16 16:58:55.939822 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:55.939459 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75998hh8" event={"ID":"4ed08f90-c844-43f1-9c65-ab8bede0ee73","Type":"ContainerDied","Data":"4be9e7863c464a5ad71a9bd2b459fa4f3cbd7f437f16c0f9befc5ac3b93a4656"} Jul 16 16:58:55.941536 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:55.941512 2571 generic.go:356] "Generic (PLEG): container finished" podID="a9b19b46-70a5-46be-b697-dcd420eed74c" containerID="6e9d769d575743f5f552ba1c57a211c6ab180a316b00cc60cfc9ab1214e9bc77" exitCode=0 Jul 16 16:58:55.941688 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:55.941593 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rk7h4" event={"ID":"a9b19b46-70a5-46be-b697-dcd420eed74c","Type":"ContainerDied","Data":"6e9d769d575743f5f552ba1c57a211c6ab180a316b00cc60cfc9ab1214e9bc77"} Jul 16 16:58:55.943442 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:55.943424 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pfl45" event={"ID":"8df978d6-8d77-40cf-a714-af26edb260f8","Type":"ContainerStarted","Data":"e172f860ef071a46c0f59bb8b61c9b671698d2b11e0090390263f5878e7864e0"} Jul 16 16:58:55.945657 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:55.945626 2571 generic.go:356] "Generic (PLEG): container finished" podID="04adff86-9c0b-495e-b73c-29da5b579fda" containerID="07b98273857ab610853d8d1212bcf6975075d27457cf0a8ce4b7571957c16a26" exitCode=0 Jul 16 16:58:55.945657 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:55.945660 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09psp4" event={"ID":"04adff86-9c0b-495e-b73c-29da5b579fda","Type":"ContainerDied","Data":"07b98273857ab610853d8d1212bcf6975075d27457cf0a8ce4b7571957c16a26"} Jul 16 16:58:56.952144 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:56.952111 2571 generic.go:356] "Generic (PLEG): container finished" podID="4ed08f90-c844-43f1-9c65-ab8bede0ee73" containerID="2d3b42b66ec8f93ea4bd15622e587af2b18fc1ad648a9f799d7f3c39fa63dca3" exitCode=0 Jul 16 16:58:56.952583 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:56.952202 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75998hh8" event={"ID":"4ed08f90-c844-43f1-9c65-ab8bede0ee73","Type":"ContainerDied","Data":"2d3b42b66ec8f93ea4bd15622e587af2b18fc1ad648a9f799d7f3c39fa63dca3"} Jul 16 16:58:56.954158 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:56.954131 2571 generic.go:356] "Generic (PLEG): container finished" podID="a9b19b46-70a5-46be-b697-dcd420eed74c" containerID="6307e9ef0a8989e27ba380a22698b555b62e70f8e7baeb7284a3536656ba5767" exitCode=0 Jul 16 16:58:56.954250 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:56.954215 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rk7h4" event={"ID":"a9b19b46-70a5-46be-b697-dcd420eed74c","Type":"ContainerDied","Data":"6307e9ef0a8989e27ba380a22698b555b62e70f8e7baeb7284a3536656ba5767"} Jul 16 16:58:56.955680 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:56.955660 2571 generic.go:356] "Generic (PLEG): container finished" podID="8df978d6-8d77-40cf-a714-af26edb260f8" containerID="e172f860ef071a46c0f59bb8b61c9b671698d2b11e0090390263f5878e7864e0" exitCode=0 Jul 16 16:58:56.955782 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:56.955748 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pfl45" event={"ID":"8df978d6-8d77-40cf-a714-af26edb260f8","Type":"ContainerDied","Data":"e172f860ef071a46c0f59bb8b61c9b671698d2b11e0090390263f5878e7864e0"} Jul 16 16:58:57.095168 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:57.095139 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09psp4" Jul 16 16:58:57.185205 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:57.185152 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/04adff86-9c0b-495e-b73c-29da5b579fda-util\") pod \"04adff86-9c0b-495e-b73c-29da5b579fda\" (UID: \"04adff86-9c0b-495e-b73c-29da5b579fda\") " Jul 16 16:58:57.185376 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:57.185233 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/04adff86-9c0b-495e-b73c-29da5b579fda-bundle\") pod \"04adff86-9c0b-495e-b73c-29da5b579fda\" (UID: \"04adff86-9c0b-495e-b73c-29da5b579fda\") " Jul 16 16:58:57.185376 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:57.185265 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gpv78\" (UniqueName: \"kubernetes.io/projected/04adff86-9c0b-495e-b73c-29da5b579fda-kube-api-access-gpv78\") pod \"04adff86-9c0b-495e-b73c-29da5b579fda\" (UID: \"04adff86-9c0b-495e-b73c-29da5b579fda\") " Jul 16 16:58:57.185785 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:57.185753 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04adff86-9c0b-495e-b73c-29da5b579fda-bundle" (OuterVolumeSpecName: "bundle") pod "04adff86-9c0b-495e-b73c-29da5b579fda" (UID: "04adff86-9c0b-495e-b73c-29da5b579fda"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:58:57.187535 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:57.187513 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04adff86-9c0b-495e-b73c-29da5b579fda-kube-api-access-gpv78" (OuterVolumeSpecName: "kube-api-access-gpv78") pod "04adff86-9c0b-495e-b73c-29da5b579fda" (UID: "04adff86-9c0b-495e-b73c-29da5b579fda"). InnerVolumeSpecName "kube-api-access-gpv78". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:58:57.191065 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:57.191044 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04adff86-9c0b-495e-b73c-29da5b579fda-util" (OuterVolumeSpecName: "util") pod "04adff86-9c0b-495e-b73c-29da5b579fda" (UID: "04adff86-9c0b-495e-b73c-29da5b579fda"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:58:57.286766 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:57.286624 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/04adff86-9c0b-495e-b73c-29da5b579fda-bundle\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:58:57.286766 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:57.286672 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gpv78\" (UniqueName: \"kubernetes.io/projected/04adff86-9c0b-495e-b73c-29da5b579fda-kube-api-access-gpv78\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:58:57.286766 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:57.286684 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/04adff86-9c0b-495e-b73c-29da5b579fda-util\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:58:57.962369 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:57.962324 2571 generic.go:356] "Generic (PLEG): container finished" podID="8df978d6-8d77-40cf-a714-af26edb260f8" containerID="b0bbfe47b299856b1d92799140adb344c0001a3a9ebb280db22d9bf8994e55a5" exitCode=0 Jul 16 16:58:57.962780 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:57.962429 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pfl45" event={"ID":"8df978d6-8d77-40cf-a714-af26edb260f8","Type":"ContainerDied","Data":"b0bbfe47b299856b1d92799140adb344c0001a3a9ebb280db22d9bf8994e55a5"} Jul 16 16:58:57.964199 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:57.964180 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09psp4" Jul 16 16:58:57.964313 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:57.964214 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09psp4" event={"ID":"04adff86-9c0b-495e-b73c-29da5b579fda","Type":"ContainerDied","Data":"2affd82acffa5243a1eae4baca8ff97350f8f915f8f030d4c34a102bad51df7e"} Jul 16 16:58:57.964313 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:57.964252 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2affd82acffa5243a1eae4baca8ff97350f8f915f8f030d4c34a102bad51df7e" Jul 16 16:58:58.098546 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:58.098523 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rk7h4" Jul 16 16:58:58.134230 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:58.134202 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75998hh8" Jul 16 16:58:58.194914 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:58.194879 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ed08f90-c844-43f1-9c65-ab8bede0ee73-bundle\") pod \"4ed08f90-c844-43f1-9c65-ab8bede0ee73\" (UID: \"4ed08f90-c844-43f1-9c65-ab8bede0ee73\") " Jul 16 16:58:58.195097 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:58.194923 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9b19b46-70a5-46be-b697-dcd420eed74c-util\") pod \"a9b19b46-70a5-46be-b697-dcd420eed74c\" (UID: \"a9b19b46-70a5-46be-b697-dcd420eed74c\") " Jul 16 16:58:58.195097 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:58.194956 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lmnhx\" (UniqueName: \"kubernetes.io/projected/a9b19b46-70a5-46be-b697-dcd420eed74c-kube-api-access-lmnhx\") pod \"a9b19b46-70a5-46be-b697-dcd420eed74c\" (UID: \"a9b19b46-70a5-46be-b697-dcd420eed74c\") " Jul 16 16:58:58.195097 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:58.194976 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9b19b46-70a5-46be-b697-dcd420eed74c-bundle\") pod \"a9b19b46-70a5-46be-b697-dcd420eed74c\" (UID: \"a9b19b46-70a5-46be-b697-dcd420eed74c\") " Jul 16 16:58:58.195097 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:58.195003 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ed08f90-c844-43f1-9c65-ab8bede0ee73-util\") pod \"4ed08f90-c844-43f1-9c65-ab8bede0ee73\" (UID: \"4ed08f90-c844-43f1-9c65-ab8bede0ee73\") " Jul 16 16:58:58.195097 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:58.195032 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fm7t5\" (UniqueName: \"kubernetes.io/projected/4ed08f90-c844-43f1-9c65-ab8bede0ee73-kube-api-access-fm7t5\") pod \"4ed08f90-c844-43f1-9c65-ab8bede0ee73\" (UID: \"4ed08f90-c844-43f1-9c65-ab8bede0ee73\") " Jul 16 16:58:58.195450 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:58.195408 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ed08f90-c844-43f1-9c65-ab8bede0ee73-bundle" (OuterVolumeSpecName: "bundle") pod "4ed08f90-c844-43f1-9c65-ab8bede0ee73" (UID: "4ed08f90-c844-43f1-9c65-ab8bede0ee73"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:58:58.195717 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:58.195692 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9b19b46-70a5-46be-b697-dcd420eed74c-bundle" (OuterVolumeSpecName: "bundle") pod "a9b19b46-70a5-46be-b697-dcd420eed74c" (UID: "a9b19b46-70a5-46be-b697-dcd420eed74c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:58:58.197190 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:58.197141 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ed08f90-c844-43f1-9c65-ab8bede0ee73-kube-api-access-fm7t5" (OuterVolumeSpecName: "kube-api-access-fm7t5") pod "4ed08f90-c844-43f1-9c65-ab8bede0ee73" (UID: "4ed08f90-c844-43f1-9c65-ab8bede0ee73"). InnerVolumeSpecName "kube-api-access-fm7t5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:58:58.197279 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:58.197193 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9b19b46-70a5-46be-b697-dcd420eed74c-kube-api-access-lmnhx" (OuterVolumeSpecName: "kube-api-access-lmnhx") pod "a9b19b46-70a5-46be-b697-dcd420eed74c" (UID: "a9b19b46-70a5-46be-b697-dcd420eed74c"). InnerVolumeSpecName "kube-api-access-lmnhx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:58:58.200443 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:58.200422 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ed08f90-c844-43f1-9c65-ab8bede0ee73-util" (OuterVolumeSpecName: "util") pod "4ed08f90-c844-43f1-9c65-ab8bede0ee73" (UID: "4ed08f90-c844-43f1-9c65-ab8bede0ee73"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:58:58.203151 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:58.203124 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9b19b46-70a5-46be-b697-dcd420eed74c-util" (OuterVolumeSpecName: "util") pod "a9b19b46-70a5-46be-b697-dcd420eed74c" (UID: "a9b19b46-70a5-46be-b697-dcd420eed74c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:58:58.296407 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:58.296311 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ed08f90-c844-43f1-9c65-ab8bede0ee73-bundle\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:58:58.296407 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:58.296341 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9b19b46-70a5-46be-b697-dcd420eed74c-util\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:58:58.296407 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:58.296353 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmnhx\" (UniqueName: \"kubernetes.io/projected/a9b19b46-70a5-46be-b697-dcd420eed74c-kube-api-access-lmnhx\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:58:58.296407 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:58.296366 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9b19b46-70a5-46be-b697-dcd420eed74c-bundle\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:58:58.296407 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:58.296376 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ed08f90-c844-43f1-9c65-ab8bede0ee73-util\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:58:58.296407 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:58.296386 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fm7t5\" (UniqueName: \"kubernetes.io/projected/4ed08f90-c844-43f1-9c65-ab8bede0ee73-kube-api-access-fm7t5\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:58:58.971078 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:58.971046 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rk7h4" Jul 16 16:58:58.971078 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:58.971069 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rk7h4" event={"ID":"a9b19b46-70a5-46be-b697-dcd420eed74c","Type":"ContainerDied","Data":"8fd97caf4ec2b7f7d92ed2db1fb5bccb5584835640a47658532625373110da51"} Jul 16 16:58:58.971584 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:58.971117 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8fd97caf4ec2b7f7d92ed2db1fb5bccb5584835640a47658532625373110da51" Jul 16 16:58:58.973006 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:58.972960 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75998hh8" event={"ID":"4ed08f90-c844-43f1-9c65-ab8bede0ee73","Type":"ContainerDied","Data":"faab75854df93b1518167ba1ceb4f2f34898f9bcdacc6884f904d5ac1efd4048"} Jul 16 16:58:58.973006 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:58.972997 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75998hh8" Jul 16 16:58:58.973219 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:58.973023 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="faab75854df93b1518167ba1ceb4f2f34898f9bcdacc6884f904d5ac1efd4048" Jul 16 16:58:59.111172 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:59.111144 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pfl45" Jul 16 16:58:59.203692 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:59.203652 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8df978d6-8d77-40cf-a714-af26edb260f8-util\") pod \"8df978d6-8d77-40cf-a714-af26edb260f8\" (UID: \"8df978d6-8d77-40cf-a714-af26edb260f8\") " Jul 16 16:58:59.203892 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:59.203712 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9957t\" (UniqueName: \"kubernetes.io/projected/8df978d6-8d77-40cf-a714-af26edb260f8-kube-api-access-9957t\") pod \"8df978d6-8d77-40cf-a714-af26edb260f8\" (UID: \"8df978d6-8d77-40cf-a714-af26edb260f8\") " Jul 16 16:58:59.203892 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:59.203755 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8df978d6-8d77-40cf-a714-af26edb260f8-bundle\") pod \"8df978d6-8d77-40cf-a714-af26edb260f8\" (UID: \"8df978d6-8d77-40cf-a714-af26edb260f8\") " Jul 16 16:58:59.204323 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:59.204285 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8df978d6-8d77-40cf-a714-af26edb260f8-bundle" (OuterVolumeSpecName: "bundle") pod "8df978d6-8d77-40cf-a714-af26edb260f8" (UID: "8df978d6-8d77-40cf-a714-af26edb260f8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:58:59.205967 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:59.205944 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8df978d6-8d77-40cf-a714-af26edb260f8-kube-api-access-9957t" (OuterVolumeSpecName: "kube-api-access-9957t") pod "8df978d6-8d77-40cf-a714-af26edb260f8" (UID: "8df978d6-8d77-40cf-a714-af26edb260f8"). InnerVolumeSpecName "kube-api-access-9957t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:58:59.209510 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:59.209471 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8df978d6-8d77-40cf-a714-af26edb260f8-util" (OuterVolumeSpecName: "util") pod "8df978d6-8d77-40cf-a714-af26edb260f8" (UID: "8df978d6-8d77-40cf-a714-af26edb260f8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:58:59.304668 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:59.304630 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8df978d6-8d77-40cf-a714-af26edb260f8-bundle\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:58:59.304668 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:59.304662 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8df978d6-8d77-40cf-a714-af26edb260f8-util\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:58:59.304668 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:59.304671 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9957t\" (UniqueName: \"kubernetes.io/projected/8df978d6-8d77-40cf-a714-af26edb260f8-kube-api-access-9957t\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:58:59.979441 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:59.979401 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pfl45" event={"ID":"8df978d6-8d77-40cf-a714-af26edb260f8","Type":"ContainerDied","Data":"61c4f81d10a864966c9dce075562fa6b98f176644356c5cf268bdea33e34cd4c"} Jul 16 16:58:59.979441 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:59.979419 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pfl45" Jul 16 16:58:59.979877 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:58:59.979451 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="61c4f81d10a864966c9dce075562fa6b98f176644356c5cf268bdea33e34cd4c" Jul 16 16:59:02.325625 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.325596 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5b6cbd7f5f-f2mxc"] Jul 16 16:59:02.326184 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.326108 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9b19b46-70a5-46be-b697-dcd420eed74c" containerName="util" Jul 16 16:59:02.326184 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.326126 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9b19b46-70a5-46be-b697-dcd420eed74c" containerName="util" Jul 16 16:59:02.326184 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.326175 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8df978d6-8d77-40cf-a714-af26edb260f8" containerName="extract" Jul 16 16:59:02.326369 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.326189 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8df978d6-8d77-40cf-a714-af26edb260f8" containerName="extract" Jul 16 16:59:02.326369 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.326203 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9b19b46-70a5-46be-b697-dcd420eed74c" containerName="extract" Jul 16 16:59:02.326369 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.326211 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9b19b46-70a5-46be-b697-dcd420eed74c" containerName="extract" Jul 16 16:59:02.326369 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.326225 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04adff86-9c0b-495e-b73c-29da5b579fda" containerName="util" Jul 16 16:59:02.326369 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.326234 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="04adff86-9c0b-495e-b73c-29da5b579fda" containerName="util" Jul 16 16:59:02.326369 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.326242 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04adff86-9c0b-495e-b73c-29da5b579fda" containerName="extract" Jul 16 16:59:02.326369 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.326251 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="04adff86-9c0b-495e-b73c-29da5b579fda" containerName="extract" Jul 16 16:59:02.326369 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.326266 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ed08f90-c844-43f1-9c65-ab8bede0ee73" containerName="util" Jul 16 16:59:02.326369 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.326274 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ed08f90-c844-43f1-9c65-ab8bede0ee73" containerName="util" Jul 16 16:59:02.326369 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.326288 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9b19b46-70a5-46be-b697-dcd420eed74c" containerName="pull" Jul 16 16:59:02.326369 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.326296 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9b19b46-70a5-46be-b697-dcd420eed74c" containerName="pull" Jul 16 16:59:02.326369 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.326308 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8df978d6-8d77-40cf-a714-af26edb260f8" containerName="util" Jul 16 16:59:02.326369 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.326316 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8df978d6-8d77-40cf-a714-af26edb260f8" containerName="util" Jul 16 16:59:02.326369 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.326357 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ed08f90-c844-43f1-9c65-ab8bede0ee73" containerName="pull" Jul 16 16:59:02.326369 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.326365 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ed08f90-c844-43f1-9c65-ab8bede0ee73" containerName="pull" Jul 16 16:59:02.326369 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.326376 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8df978d6-8d77-40cf-a714-af26edb260f8" containerName="pull" Jul 16 16:59:02.326369 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.326383 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8df978d6-8d77-40cf-a714-af26edb260f8" containerName="pull" Jul 16 16:59:02.327115 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.326398 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ed08f90-c844-43f1-9c65-ab8bede0ee73" containerName="extract" Jul 16 16:59:02.327115 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.326406 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ed08f90-c844-43f1-9c65-ab8bede0ee73" containerName="extract" Jul 16 16:59:02.327115 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.326419 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04adff86-9c0b-495e-b73c-29da5b579fda" containerName="pull" Jul 16 16:59:02.327115 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.326427 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="04adff86-9c0b-495e-b73c-29da5b579fda" containerName="pull" Jul 16 16:59:02.327115 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.326512 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9b19b46-70a5-46be-b697-dcd420eed74c" containerName="extract" Jul 16 16:59:02.327115 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.326525 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="8df978d6-8d77-40cf-a714-af26edb260f8" containerName="extract" Jul 16 16:59:02.327115 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.326534 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ed08f90-c844-43f1-9c65-ab8bede0ee73" containerName="extract" Jul 16 16:59:02.327115 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.326546 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="04adff86-9c0b-495e-b73c-29da5b579fda" containerName="extract" Jul 16 16:59:02.330038 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.330014 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b6cbd7f5f-f2mxc" Jul 16 16:59:02.377644 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.377614 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b6cbd7f5f-f2mxc"] Jul 16 16:59:02.428764 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.428712 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4178dcc7-bf24-473e-b3f3-05e77c83e3a2-service-ca\") pod \"console-5b6cbd7f5f-f2mxc\" (UID: \"4178dcc7-bf24-473e-b3f3-05e77c83e3a2\") " pod="openshift-console/console-5b6cbd7f5f-f2mxc" Jul 16 16:59:02.428872 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.428774 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4178dcc7-bf24-473e-b3f3-05e77c83e3a2-trusted-ca-bundle\") pod \"console-5b6cbd7f5f-f2mxc\" (UID: \"4178dcc7-bf24-473e-b3f3-05e77c83e3a2\") " pod="openshift-console/console-5b6cbd7f5f-f2mxc" Jul 16 16:59:02.428872 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.428797 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhl7f\" (UniqueName: \"kubernetes.io/projected/4178dcc7-bf24-473e-b3f3-05e77c83e3a2-kube-api-access-mhl7f\") pod \"console-5b6cbd7f5f-f2mxc\" (UID: \"4178dcc7-bf24-473e-b3f3-05e77c83e3a2\") " pod="openshift-console/console-5b6cbd7f5f-f2mxc" Jul 16 16:59:02.428872 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.428824 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4178dcc7-bf24-473e-b3f3-05e77c83e3a2-console-serving-cert\") pod \"console-5b6cbd7f5f-f2mxc\" (UID: \"4178dcc7-bf24-473e-b3f3-05e77c83e3a2\") " pod="openshift-console/console-5b6cbd7f5f-f2mxc" Jul 16 16:59:02.428872 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.428837 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4178dcc7-bf24-473e-b3f3-05e77c83e3a2-console-config\") pod \"console-5b6cbd7f5f-f2mxc\" (UID: \"4178dcc7-bf24-473e-b3f3-05e77c83e3a2\") " pod="openshift-console/console-5b6cbd7f5f-f2mxc" Jul 16 16:59:02.428990 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.428932 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4178dcc7-bf24-473e-b3f3-05e77c83e3a2-oauth-serving-cert\") pod \"console-5b6cbd7f5f-f2mxc\" (UID: \"4178dcc7-bf24-473e-b3f3-05e77c83e3a2\") " pod="openshift-console/console-5b6cbd7f5f-f2mxc" Jul 16 16:59:02.428990 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.428972 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4178dcc7-bf24-473e-b3f3-05e77c83e3a2-console-oauth-config\") pod \"console-5b6cbd7f5f-f2mxc\" (UID: \"4178dcc7-bf24-473e-b3f3-05e77c83e3a2\") " pod="openshift-console/console-5b6cbd7f5f-f2mxc" Jul 16 16:59:02.529500 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.529469 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4178dcc7-bf24-473e-b3f3-05e77c83e3a2-console-oauth-config\") pod \"console-5b6cbd7f5f-f2mxc\" (UID: \"4178dcc7-bf24-473e-b3f3-05e77c83e3a2\") " pod="openshift-console/console-5b6cbd7f5f-f2mxc" Jul 16 16:59:02.529608 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.529515 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4178dcc7-bf24-473e-b3f3-05e77c83e3a2-service-ca\") pod \"console-5b6cbd7f5f-f2mxc\" (UID: \"4178dcc7-bf24-473e-b3f3-05e77c83e3a2\") " pod="openshift-console/console-5b6cbd7f5f-f2mxc" Jul 16 16:59:02.529608 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.529538 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4178dcc7-bf24-473e-b3f3-05e77c83e3a2-trusted-ca-bundle\") pod \"console-5b6cbd7f5f-f2mxc\" (UID: \"4178dcc7-bf24-473e-b3f3-05e77c83e3a2\") " pod="openshift-console/console-5b6cbd7f5f-f2mxc" Jul 16 16:59:02.529823 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.529796 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhl7f\" (UniqueName: \"kubernetes.io/projected/4178dcc7-bf24-473e-b3f3-05e77c83e3a2-kube-api-access-mhl7f\") pod \"console-5b6cbd7f5f-f2mxc\" (UID: \"4178dcc7-bf24-473e-b3f3-05e77c83e3a2\") " pod="openshift-console/console-5b6cbd7f5f-f2mxc" Jul 16 16:59:02.529904 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.529858 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4178dcc7-bf24-473e-b3f3-05e77c83e3a2-console-serving-cert\") pod \"console-5b6cbd7f5f-f2mxc\" (UID: \"4178dcc7-bf24-473e-b3f3-05e77c83e3a2\") " pod="openshift-console/console-5b6cbd7f5f-f2mxc" Jul 16 16:59:02.529904 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.529885 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4178dcc7-bf24-473e-b3f3-05e77c83e3a2-console-config\") pod \"console-5b6cbd7f5f-f2mxc\" (UID: \"4178dcc7-bf24-473e-b3f3-05e77c83e3a2\") " pod="openshift-console/console-5b6cbd7f5f-f2mxc" Jul 16 16:59:02.530022 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.529950 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4178dcc7-bf24-473e-b3f3-05e77c83e3a2-oauth-serving-cert\") pod \"console-5b6cbd7f5f-f2mxc\" (UID: \"4178dcc7-bf24-473e-b3f3-05e77c83e3a2\") " pod="openshift-console/console-5b6cbd7f5f-f2mxc" Jul 16 16:59:02.530224 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.530199 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4178dcc7-bf24-473e-b3f3-05e77c83e3a2-service-ca\") pod \"console-5b6cbd7f5f-f2mxc\" (UID: \"4178dcc7-bf24-473e-b3f3-05e77c83e3a2\") " pod="openshift-console/console-5b6cbd7f5f-f2mxc" Jul 16 16:59:02.530452 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.530433 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4178dcc7-bf24-473e-b3f3-05e77c83e3a2-trusted-ca-bundle\") pod \"console-5b6cbd7f5f-f2mxc\" (UID: \"4178dcc7-bf24-473e-b3f3-05e77c83e3a2\") " pod="openshift-console/console-5b6cbd7f5f-f2mxc" Jul 16 16:59:02.530622 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.530602 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4178dcc7-bf24-473e-b3f3-05e77c83e3a2-console-config\") pod \"console-5b6cbd7f5f-f2mxc\" (UID: \"4178dcc7-bf24-473e-b3f3-05e77c83e3a2\") " pod="openshift-console/console-5b6cbd7f5f-f2mxc" Jul 16 16:59:02.530657 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.530602 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4178dcc7-bf24-473e-b3f3-05e77c83e3a2-oauth-serving-cert\") pod \"console-5b6cbd7f5f-f2mxc\" (UID: \"4178dcc7-bf24-473e-b3f3-05e77c83e3a2\") " pod="openshift-console/console-5b6cbd7f5f-f2mxc" Jul 16 16:59:02.532057 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.532037 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4178dcc7-bf24-473e-b3f3-05e77c83e3a2-console-oauth-config\") pod \"console-5b6cbd7f5f-f2mxc\" (UID: \"4178dcc7-bf24-473e-b3f3-05e77c83e3a2\") " pod="openshift-console/console-5b6cbd7f5f-f2mxc" Jul 16 16:59:02.532181 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.532161 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4178dcc7-bf24-473e-b3f3-05e77c83e3a2-console-serving-cert\") pod \"console-5b6cbd7f5f-f2mxc\" (UID: \"4178dcc7-bf24-473e-b3f3-05e77c83e3a2\") " pod="openshift-console/console-5b6cbd7f5f-f2mxc" Jul 16 16:59:02.546403 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.546374 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhl7f\" (UniqueName: \"kubernetes.io/projected/4178dcc7-bf24-473e-b3f3-05e77c83e3a2-kube-api-access-mhl7f\") pod \"console-5b6cbd7f5f-f2mxc\" (UID: \"4178dcc7-bf24-473e-b3f3-05e77c83e3a2\") " pod="openshift-console/console-5b6cbd7f5f-f2mxc" Jul 16 16:59:02.640038 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.639983 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b6cbd7f5f-f2mxc" Jul 16 16:59:02.779171 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.779147 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b6cbd7f5f-f2mxc"] Jul 16 16:59:02.780055 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:59:02.780036 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4178dcc7_bf24_473e_b3f3_05e77c83e3a2.slice/crio-77bd01654487c8c339c1ed6f8e9f6d60bb40fc6dd8401101a369d5300607a13d WatchSource:0}: Error finding container 77bd01654487c8c339c1ed6f8e9f6d60bb40fc6dd8401101a369d5300607a13d: Status 404 returned error can't find the container with id 77bd01654487c8c339c1ed6f8e9f6d60bb40fc6dd8401101a369d5300607a13d Jul 16 16:59:02.996753 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.996708 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b6cbd7f5f-f2mxc" event={"ID":"4178dcc7-bf24-473e-b3f3-05e77c83e3a2","Type":"ContainerStarted","Data":"a61aad4bfc51ba7bf0746277b0688e0dc2cdf18652548fbefc8adfd060f7f46c"} Jul 16 16:59:02.996895 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:02.996765 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b6cbd7f5f-f2mxc" event={"ID":"4178dcc7-bf24-473e-b3f3-05e77c83e3a2","Type":"ContainerStarted","Data":"77bd01654487c8c339c1ed6f8e9f6d60bb40fc6dd8401101a369d5300607a13d"} Jul 16 16:59:03.039906 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:03.039863 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5b6cbd7f5f-f2mxc" podStartSLOduration=1.039849662 podStartE2EDuration="1.039849662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:59:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:59:03.039665199 +0000 UTC m=+494.781240494" watchObservedRunningTime="2026-07-16 16:59:03.039849662 +0000 UTC m=+494.781424957" Jul 16 16:59:12.640256 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:12.640218 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5b6cbd7f5f-f2mxc" Jul 16 16:59:12.640669 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:12.640271 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5b6cbd7f5f-f2mxc" Jul 16 16:59:12.645270 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:12.645238 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5b6cbd7f5f-f2mxc" Jul 16 16:59:13.052319 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:13.052295 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5b6cbd7f5f-f2mxc" Jul 16 16:59:13.124495 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:13.124463 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9dd7fc48d-gj566"] Jul 16 16:59:18.125285 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:18.125247 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-ck96v"] Jul 16 16:59:18.128881 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:18.128856 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ck96v" Jul 16 16:59:18.132504 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:18.132483 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 16 16:59:18.133513 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:18.133485 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-lnn5l\"" Jul 16 16:59:18.133612 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:18.133493 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 16 16:59:18.146165 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:18.146142 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-ck96v"] Jul 16 16:59:18.248687 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:18.248664 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2nm5\" (UniqueName: \"kubernetes.io/projected/14df11f0-5fc9-491e-8e4a-54867cb706b0-kube-api-access-d2nm5\") pod \"authorino-operator-75cc8d5cb7-ck96v\" (UID: \"14df11f0-5fc9-491e-8e4a-54867cb706b0\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-ck96v" Jul 16 16:59:18.349399 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:18.349374 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2nm5\" (UniqueName: \"kubernetes.io/projected/14df11f0-5fc9-491e-8e4a-54867cb706b0-kube-api-access-d2nm5\") pod \"authorino-operator-75cc8d5cb7-ck96v\" (UID: \"14df11f0-5fc9-491e-8e4a-54867cb706b0\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-ck96v" Jul 16 16:59:18.371012 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:18.370984 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2nm5\" (UniqueName: \"kubernetes.io/projected/14df11f0-5fc9-491e-8e4a-54867cb706b0-kube-api-access-d2nm5\") pod \"authorino-operator-75cc8d5cb7-ck96v\" (UID: \"14df11f0-5fc9-491e-8e4a-54867cb706b0\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-ck96v" Jul 16 16:59:18.440617 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:18.440595 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ck96v" Jul 16 16:59:18.581065 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:18.581032 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-ck96v"] Jul 16 16:59:18.582144 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:59:18.582112 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14df11f0_5fc9_491e_8e4a_54867cb706b0.slice/crio-81832f4da99ebbd5085ef9ef1e5c05478d662ba94dd821e2b2238c2d657ee024 WatchSource:0}: Error finding container 81832f4da99ebbd5085ef9ef1e5c05478d662ba94dd821e2b2238c2d657ee024: Status 404 returned error can't find the container with id 81832f4da99ebbd5085ef9ef1e5c05478d662ba94dd821e2b2238c2d657ee024 Jul 16 16:59:19.082678 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:19.082647 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ck96v" event={"ID":"14df11f0-5fc9-491e-8e4a-54867cb706b0","Type":"ContainerStarted","Data":"81832f4da99ebbd5085ef9ef1e5c05478d662ba94dd821e2b2238c2d657ee024"} Jul 16 16:59:23.105223 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:23.105186 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ck96v" event={"ID":"14df11f0-5fc9-491e-8e4a-54867cb706b0","Type":"ContainerStarted","Data":"1d4e91348c0ca41ba27faeceb108c8304bc1af7c6c8ab00da21cfd0ec63ea1fb"} Jul 16 16:59:23.105617 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:23.105267 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ck96v" Jul 16 16:59:34.113225 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:34.113189 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ck96v" Jul 16 16:59:34.172291 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:34.172243 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ck96v" podStartSLOduration=12.102308171 podStartE2EDuration="16.172226906s" totalImagesPullingTime="4.069918735s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:59:23.130791682 +0000 UTC m=+514.872366976" watchObservedRunningTime="2026-07-16 16:59:34.172226906 +0000 UTC m=+525.913802261" Jul 16 16:59:36.765908 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:36.763333 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfw8r"] Jul 16 16:59:36.769753 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:36.769713 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfw8r" Jul 16 16:59:36.774040 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:36.774018 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-tjtrx\"" Jul 16 16:59:36.786521 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:36.786492 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfw8r"] Jul 16 16:59:36.887291 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:36.887265 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ce787c88-f7cc-452f-9989-3199cb2353ee-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bfw8r\" (UID: \"ce787c88-f7cc-452f-9989-3199cb2353ee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfw8r" Jul 16 16:59:36.887394 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:36.887330 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpql5\" (UniqueName: \"kubernetes.io/projected/ce787c88-f7cc-452f-9989-3199cb2353ee-kube-api-access-mpql5\") pod \"kuadrant-operator-controller-manager-86878c769f-bfw8r\" (UID: \"ce787c88-f7cc-452f-9989-3199cb2353ee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfw8r" Jul 16 16:59:36.987667 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:36.987647 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ce787c88-f7cc-452f-9989-3199cb2353ee-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bfw8r\" (UID: \"ce787c88-f7cc-452f-9989-3199cb2353ee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfw8r" Jul 16 16:59:36.987781 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:36.987708 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpql5\" (UniqueName: \"kubernetes.io/projected/ce787c88-f7cc-452f-9989-3199cb2353ee-kube-api-access-mpql5\") pod \"kuadrant-operator-controller-manager-86878c769f-bfw8r\" (UID: \"ce787c88-f7cc-452f-9989-3199cb2353ee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfw8r" Jul 16 16:59:36.988049 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:36.988031 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ce787c88-f7cc-452f-9989-3199cb2353ee-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bfw8r\" (UID: \"ce787c88-f7cc-452f-9989-3199cb2353ee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfw8r" Jul 16 16:59:37.006879 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:37.006858 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpql5\" (UniqueName: \"kubernetes.io/projected/ce787c88-f7cc-452f-9989-3199cb2353ee-kube-api-access-mpql5\") pod \"kuadrant-operator-controller-manager-86878c769f-bfw8r\" (UID: \"ce787c88-f7cc-452f-9989-3199cb2353ee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfw8r" Jul 16 16:59:37.080604 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:37.080556 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfw8r" Jul 16 16:59:37.241704 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:37.241680 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfw8r"] Jul 16 16:59:37.244408 ip-10-0-128-50 kubenswrapper[2571]: W0716 16:59:37.244373 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce787c88_f7cc_452f_9989_3199cb2353ee.slice/crio-e70c6d93aa4b83f8a54cb40c032c3f6d3326e41d50dbe88bdb40437eef9c5cbf WatchSource:0}: Error finding container e70c6d93aa4b83f8a54cb40c032c3f6d3326e41d50dbe88bdb40437eef9c5cbf: Status 404 returned error can't find the container with id e70c6d93aa4b83f8a54cb40c032c3f6d3326e41d50dbe88bdb40437eef9c5cbf Jul 16 16:59:38.150859 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:38.150813 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-9dd7fc48d-gj566" podUID="6e52ccea-4b86-433c-b6aa-cbe0fa0870a6" containerName="console" containerID="cri-o://4b7582a111360b38f99bea72882ce5cee587c6667118c31a6275015aad58c477" gracePeriod=15 Jul 16 16:59:38.182109 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:38.182072 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfw8r" event={"ID":"ce787c88-f7cc-452f-9989-3199cb2353ee","Type":"ContainerStarted","Data":"e70c6d93aa4b83f8a54cb40c032c3f6d3326e41d50dbe88bdb40437eef9c5cbf"} Jul 16 16:59:38.504918 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:38.504889 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9dd7fc48d-gj566" Jul 16 16:59:38.602310 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:38.602269 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qttxc\" (UniqueName: \"kubernetes.io/projected/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-kube-api-access-qttxc\") pod \"6e52ccea-4b86-433c-b6aa-cbe0fa0870a6\" (UID: \"6e52ccea-4b86-433c-b6aa-cbe0fa0870a6\") " Jul 16 16:59:38.602502 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:38.602342 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-console-oauth-config\") pod \"6e52ccea-4b86-433c-b6aa-cbe0fa0870a6\" (UID: \"6e52ccea-4b86-433c-b6aa-cbe0fa0870a6\") " Jul 16 16:59:38.602502 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:38.602390 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-oauth-serving-cert\") pod \"6e52ccea-4b86-433c-b6aa-cbe0fa0870a6\" (UID: \"6e52ccea-4b86-433c-b6aa-cbe0fa0870a6\") " Jul 16 16:59:38.602502 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:38.602415 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-console-serving-cert\") pod \"6e52ccea-4b86-433c-b6aa-cbe0fa0870a6\" (UID: \"6e52ccea-4b86-433c-b6aa-cbe0fa0870a6\") " Jul 16 16:59:38.602502 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:38.602439 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-service-ca\") pod \"6e52ccea-4b86-433c-b6aa-cbe0fa0870a6\" (UID: \"6e52ccea-4b86-433c-b6aa-cbe0fa0870a6\") " Jul 16 16:59:38.602502 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:38.602487 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-trusted-ca-bundle\") pod \"6e52ccea-4b86-433c-b6aa-cbe0fa0870a6\" (UID: \"6e52ccea-4b86-433c-b6aa-cbe0fa0870a6\") " Jul 16 16:59:38.602796 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:38.602521 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-console-config\") pod \"6e52ccea-4b86-433c-b6aa-cbe0fa0870a6\" (UID: \"6e52ccea-4b86-433c-b6aa-cbe0fa0870a6\") " Jul 16 16:59:38.603272 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:38.603176 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-console-config" (OuterVolumeSpecName: "console-config") pod "6e52ccea-4b86-433c-b6aa-cbe0fa0870a6" (UID: "6e52ccea-4b86-433c-b6aa-cbe0fa0870a6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 16:59:38.603693 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:38.603527 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6e52ccea-4b86-433c-b6aa-cbe0fa0870a6" (UID: "6e52ccea-4b86-433c-b6aa-cbe0fa0870a6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 16:59:38.603693 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:38.603612 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-service-ca" (OuterVolumeSpecName: "service-ca") pod "6e52ccea-4b86-433c-b6aa-cbe0fa0870a6" (UID: "6e52ccea-4b86-433c-b6aa-cbe0fa0870a6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 16:59:38.603859 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:38.603707 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6e52ccea-4b86-433c-b6aa-cbe0fa0870a6" (UID: "6e52ccea-4b86-433c-b6aa-cbe0fa0870a6"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 16:59:38.605527 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:38.605066 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6e52ccea-4b86-433c-b6aa-cbe0fa0870a6" (UID: "6e52ccea-4b86-433c-b6aa-cbe0fa0870a6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 16:59:38.605527 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:38.605472 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6e52ccea-4b86-433c-b6aa-cbe0fa0870a6" (UID: "6e52ccea-4b86-433c-b6aa-cbe0fa0870a6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 16:59:38.605527 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:38.605481 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-kube-api-access-qttxc" (OuterVolumeSpecName: "kube-api-access-qttxc") pod "6e52ccea-4b86-433c-b6aa-cbe0fa0870a6" (UID: "6e52ccea-4b86-433c-b6aa-cbe0fa0870a6"). InnerVolumeSpecName "kube-api-access-qttxc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:59:38.703324 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:38.703297 2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-console-serving-cert\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:59:38.703324 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:38.703322 2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-service-ca\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:59:38.703465 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:38.703331 2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-trusted-ca-bundle\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:59:38.703465 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:38.703338 2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-console-config\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:59:38.703465 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:38.703347 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qttxc\" (UniqueName: \"kubernetes.io/projected/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-kube-api-access-qttxc\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:59:38.703465 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:38.703355 2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-console-oauth-config\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:59:38.703465 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:38.703363 2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6-oauth-serving-cert\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 16:59:39.187923 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:39.187890 2571 generic.go:356] "Generic (PLEG): container finished" podID="6e52ccea-4b86-433c-b6aa-cbe0fa0870a6" containerID="4b7582a111360b38f99bea72882ce5cee587c6667118c31a6275015aad58c477" exitCode=0 Jul 16 16:59:39.188319 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:39.187931 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9dd7fc48d-gj566" Jul 16 16:59:39.188319 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:39.187940 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9dd7fc48d-gj566" event={"ID":"6e52ccea-4b86-433c-b6aa-cbe0fa0870a6","Type":"ContainerDied","Data":"4b7582a111360b38f99bea72882ce5cee587c6667118c31a6275015aad58c477"} Jul 16 16:59:39.188319 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:39.187977 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9dd7fc48d-gj566" event={"ID":"6e52ccea-4b86-433c-b6aa-cbe0fa0870a6","Type":"ContainerDied","Data":"da0d451b671f22a465302392c85aee0bc1166ede40ceab2681635a44125c9dae"} Jul 16 16:59:39.188319 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:39.187996 2571 scope.go:117] "RemoveContainer" containerID="4b7582a111360b38f99bea72882ce5cee587c6667118c31a6275015aad58c477" Jul 16 16:59:39.196303 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:39.196284 2571 scope.go:117] "RemoveContainer" containerID="4b7582a111360b38f99bea72882ce5cee587c6667118c31a6275015aad58c477" Jul 16 16:59:39.196539 ip-10-0-128-50 kubenswrapper[2571]: E0716 16:59:39.196522 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b7582a111360b38f99bea72882ce5cee587c6667118c31a6275015aad58c477\": container with ID starting with 4b7582a111360b38f99bea72882ce5cee587c6667118c31a6275015aad58c477 not found: ID does not exist" containerID="4b7582a111360b38f99bea72882ce5cee587c6667118c31a6275015aad58c477" Jul 16 16:59:39.196587 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:39.196545 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b7582a111360b38f99bea72882ce5cee587c6667118c31a6275015aad58c477"} err="failed to get container status \"4b7582a111360b38f99bea72882ce5cee587c6667118c31a6275015aad58c477\": rpc error: code = NotFound desc = could not find container \"4b7582a111360b38f99bea72882ce5cee587c6667118c31a6275015aad58c477\": container with ID starting with 4b7582a111360b38f99bea72882ce5cee587c6667118c31a6275015aad58c477 not found: ID does not exist" Jul 16 16:59:39.211596 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:39.211564 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9dd7fc48d-gj566"] Jul 16 16:59:39.216540 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:39.216517 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-9dd7fc48d-gj566"] Jul 16 16:59:40.784167 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:40.784129 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e52ccea-4b86-433c-b6aa-cbe0fa0870a6" path="/var/lib/kubelet/pods/6e52ccea-4b86-433c-b6aa-cbe0fa0870a6/volumes" Jul 16 16:59:42.208013 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:42.207984 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfw8r" event={"ID":"ce787c88-f7cc-452f-9989-3199cb2353ee","Type":"ContainerStarted","Data":"bc3fdc6b32447cd9e8fbdae01dca5d41a3571c6c7e86bd3e8294d88b25e0c9db"} Jul 16 16:59:42.208431 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:42.208050 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfw8r" Jul 16 16:59:42.241781 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:42.241708 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfw8r" podStartSLOduration=1.9073068050000002 podStartE2EDuration="6.241689959s" totalImagesPullingTime="4.334383154s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:59:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:59:42.237747686 +0000 UTC m=+533.979322973" watchObservedRunningTime="2026-07-16 16:59:42.241689959 +0000 UTC m=+533.983265268" Jul 16 16:59:53.215387 ip-10-0-128-50 kubenswrapper[2571]: I0716 16:59:53.215355 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfw8r" Jul 16 17:00:13.155656 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:13.155624 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-xdvlw"] Jul 16 17:00:13.156156 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:13.155956 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e52ccea-4b86-433c-b6aa-cbe0fa0870a6" containerName="console" Jul 16 17:00:13.156156 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:13.155969 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e52ccea-4b86-433c-b6aa-cbe0fa0870a6" containerName="console" Jul 16 17:00:13.156156 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:13.156037 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e52ccea-4b86-433c-b6aa-cbe0fa0870a6" containerName="console" Jul 16 17:00:13.162833 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:13.162809 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xdvlw" Jul 16 17:00:13.169743 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:13.169704 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-2srxg\"" Jul 16 17:00:13.172951 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:13.172928 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-xdvlw"] Jul 16 17:00:13.259217 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:13.259191 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhhph\" (UniqueName: \"kubernetes.io/projected/d07a06e1-dd01-4155-9295-bb7224924d85-kube-api-access-vhhph\") pod \"authorino-58cd955675-xdvlw\" (UID: \"d07a06e1-dd01-4155-9295-bb7224924d85\") " pod="kuadrant-system/authorino-58cd955675-xdvlw" Jul 16 17:00:13.360430 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:13.360403 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhhph\" (UniqueName: \"kubernetes.io/projected/d07a06e1-dd01-4155-9295-bb7224924d85-kube-api-access-vhhph\") pod \"authorino-58cd955675-xdvlw\" (UID: \"d07a06e1-dd01-4155-9295-bb7224924d85\") " pod="kuadrant-system/authorino-58cd955675-xdvlw" Jul 16 17:00:13.375632 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:13.375610 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhhph\" (UniqueName: \"kubernetes.io/projected/d07a06e1-dd01-4155-9295-bb7224924d85-kube-api-access-vhhph\") pod \"authorino-58cd955675-xdvlw\" (UID: \"d07a06e1-dd01-4155-9295-bb7224924d85\") " pod="kuadrant-system/authorino-58cd955675-xdvlw" Jul 16 17:00:13.472508 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:13.472484 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xdvlw" Jul 16 17:00:13.607793 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:13.607770 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-xdvlw"] Jul 16 17:00:13.609503 ip-10-0-128-50 kubenswrapper[2571]: W0716 17:00:13.609477 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd07a06e1_dd01_4155_9295_bb7224924d85.slice/crio-dcb241f0257a5ae7a3a53ee14823a0f8cc7229fbac6d4c36df9f0ee3c32c468b WatchSource:0}: Error finding container dcb241f0257a5ae7a3a53ee14823a0f8cc7229fbac6d4c36df9f0ee3c32c468b: Status 404 returned error can't find the container with id dcb241f0257a5ae7a3a53ee14823a0f8cc7229fbac6d4c36df9f0ee3c32c468b Jul 16 17:00:14.373505 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:14.373472 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xdvlw" event={"ID":"d07a06e1-dd01-4155-9295-bb7224924d85","Type":"ContainerStarted","Data":"dcb241f0257a5ae7a3a53ee14823a0f8cc7229fbac6d4c36df9f0ee3c32c468b"} Jul 16 17:00:18.403341 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:18.403303 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xdvlw" event={"ID":"d07a06e1-dd01-4155-9295-bb7224924d85","Type":"ContainerStarted","Data":"cc0a2ac3a85ff0fba4cef1fc5b696221137a6c2e6faea0551460f6a1bbd6818a"} Jul 16 17:00:18.429373 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:18.429325 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-xdvlw" podStartSLOduration=0.878583926 podStartE2EDuration="5.429310676s" totalImagesPullingTime="4.55072675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:00:18.426186846 +0000 UTC m=+570.167762143" watchObservedRunningTime="2026-07-16 17:00:18.429310676 +0000 UTC m=+570.170886013" Jul 16 17:00:18.564075 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:18.564036 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-xdvlw"] Jul 16 17:00:20.413637 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:20.413590 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-xdvlw" podUID="d07a06e1-dd01-4155-9295-bb7224924d85" containerName="authorino" containerID="cri-o://cc0a2ac3a85ff0fba4cef1fc5b696221137a6c2e6faea0551460f6a1bbd6818a" gracePeriod=30 Jul 16 17:00:20.654504 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:20.654483 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xdvlw" Jul 16 17:00:20.719665 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:20.719639 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vhhph\" (UniqueName: \"kubernetes.io/projected/d07a06e1-dd01-4155-9295-bb7224924d85-kube-api-access-vhhph\") pod \"d07a06e1-dd01-4155-9295-bb7224924d85\" (UID: \"d07a06e1-dd01-4155-9295-bb7224924d85\") " Jul 16 17:00:20.721522 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:20.721502 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d07a06e1-dd01-4155-9295-bb7224924d85-kube-api-access-vhhph" (OuterVolumeSpecName: "kube-api-access-vhhph") pod "d07a06e1-dd01-4155-9295-bb7224924d85" (UID: "d07a06e1-dd01-4155-9295-bb7224924d85"). InnerVolumeSpecName "kube-api-access-vhhph". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:00:20.820277 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:20.820258 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vhhph\" (UniqueName: \"kubernetes.io/projected/d07a06e1-dd01-4155-9295-bb7224924d85-kube-api-access-vhhph\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:00:21.420246 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:21.420210 2571 generic.go:356] "Generic (PLEG): container finished" podID="d07a06e1-dd01-4155-9295-bb7224924d85" containerID="cc0a2ac3a85ff0fba4cef1fc5b696221137a6c2e6faea0551460f6a1bbd6818a" exitCode=0 Jul 16 17:00:21.420628 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:21.420256 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xdvlw" Jul 16 17:00:21.420628 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:21.420297 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xdvlw" event={"ID":"d07a06e1-dd01-4155-9295-bb7224924d85","Type":"ContainerDied","Data":"cc0a2ac3a85ff0fba4cef1fc5b696221137a6c2e6faea0551460f6a1bbd6818a"} Jul 16 17:00:21.420628 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:21.420341 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xdvlw" event={"ID":"d07a06e1-dd01-4155-9295-bb7224924d85","Type":"ContainerDied","Data":"dcb241f0257a5ae7a3a53ee14823a0f8cc7229fbac6d4c36df9f0ee3c32c468b"} Jul 16 17:00:21.420628 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:21.420358 2571 scope.go:117] "RemoveContainer" containerID="cc0a2ac3a85ff0fba4cef1fc5b696221137a6c2e6faea0551460f6a1bbd6818a" Jul 16 17:00:21.429759 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:21.429719 2571 scope.go:117] "RemoveContainer" containerID="cc0a2ac3a85ff0fba4cef1fc5b696221137a6c2e6faea0551460f6a1bbd6818a" Jul 16 17:00:21.430056 ip-10-0-128-50 kubenswrapper[2571]: E0716 17:00:21.430028 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc0a2ac3a85ff0fba4cef1fc5b696221137a6c2e6faea0551460f6a1bbd6818a\": container with ID starting with cc0a2ac3a85ff0fba4cef1fc5b696221137a6c2e6faea0551460f6a1bbd6818a not found: ID does not exist" containerID="cc0a2ac3a85ff0fba4cef1fc5b696221137a6c2e6faea0551460f6a1bbd6818a" Jul 16 17:00:21.430169 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:21.430060 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc0a2ac3a85ff0fba4cef1fc5b696221137a6c2e6faea0551460f6a1bbd6818a"} err="failed to get container status \"cc0a2ac3a85ff0fba4cef1fc5b696221137a6c2e6faea0551460f6a1bbd6818a\": rpc error: code = NotFound desc = could not find container \"cc0a2ac3a85ff0fba4cef1fc5b696221137a6c2e6faea0551460f6a1bbd6818a\": container with ID starting with cc0a2ac3a85ff0fba4cef1fc5b696221137a6c2e6faea0551460f6a1bbd6818a not found: ID does not exist" Jul 16 17:00:21.441189 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:21.441166 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-xdvlw"] Jul 16 17:00:21.444109 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:21.444085 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-xdvlw"] Jul 16 17:00:22.777316 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:22.777282 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d07a06e1-dd01-4155-9295-bb7224924d85" path="/var/lib/kubelet/pods/d07a06e1-dd01-4155-9295-bb7224924d85/volumes" Jul 16 17:00:41.259376 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:41.259339 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lk4ks"] Jul 16 17:00:41.259716 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:41.259672 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d07a06e1-dd01-4155-9295-bb7224924d85" containerName="authorino" Jul 16 17:00:41.259716 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:41.259683 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d07a06e1-dd01-4155-9295-bb7224924d85" containerName="authorino" Jul 16 17:00:41.259836 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:41.259759 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d07a06e1-dd01-4155-9295-bb7224924d85" containerName="authorino" Jul 16 17:00:41.263875 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:41.263858 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-lk4ks" Jul 16 17:00:41.267584 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:41.267563 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-2srxg\"" Jul 16 17:00:41.271294 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:41.271272 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lk4ks"] Jul 16 17:00:41.369350 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:41.369326 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9p9w\" (UniqueName: \"kubernetes.io/projected/d368b5e6-24a4-4cde-9db3-b345f6969ae6-kube-api-access-m9p9w\") pod \"authorino-5d4b67c446-lk4ks\" (UID: \"d368b5e6-24a4-4cde-9db3-b345f6969ae6\") " pod="kuadrant-system/authorino-5d4b67c446-lk4ks" Jul 16 17:00:41.469766 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:41.469711 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9p9w\" (UniqueName: \"kubernetes.io/projected/d368b5e6-24a4-4cde-9db3-b345f6969ae6-kube-api-access-m9p9w\") pod \"authorino-5d4b67c446-lk4ks\" (UID: \"d368b5e6-24a4-4cde-9db3-b345f6969ae6\") " pod="kuadrant-system/authorino-5d4b67c446-lk4ks" Jul 16 17:00:41.481720 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:41.481692 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9p9w\" (UniqueName: \"kubernetes.io/projected/d368b5e6-24a4-4cde-9db3-b345f6969ae6-kube-api-access-m9p9w\") pod \"authorino-5d4b67c446-lk4ks\" (UID: \"d368b5e6-24a4-4cde-9db3-b345f6969ae6\") " pod="kuadrant-system/authorino-5d4b67c446-lk4ks" Jul 16 17:00:41.521406 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:41.521341 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lk4ks"] Jul 16 17:00:41.521556 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:41.521543 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-lk4ks" Jul 16 17:00:41.547153 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:41.547128 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6db5c47c98-spmzs"] Jul 16 17:00:41.550764 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:41.550745 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6db5c47c98-spmzs" Jul 16 17:00:41.559336 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:41.559310 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6db5c47c98-spmzs"] Jul 16 17:00:41.654049 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:41.654025 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lk4ks"] Jul 16 17:00:41.655544 ip-10-0-128-50 kubenswrapper[2571]: W0716 17:00:41.655521 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd368b5e6_24a4_4cde_9db3_b345f6969ae6.slice/crio-3601e1cbe066546a9d4d5f20a014f8c9714f44083cd6f362dcaf2ea01ebd3e40 WatchSource:0}: Error finding container 3601e1cbe066546a9d4d5f20a014f8c9714f44083cd6f362dcaf2ea01ebd3e40: Status 404 returned error can't find the container with id 3601e1cbe066546a9d4d5f20a014f8c9714f44083cd6f362dcaf2ea01ebd3e40 Jul 16 17:00:41.671604 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:41.671576 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5vlj\" (UniqueName: \"kubernetes.io/projected/e26fb045-02a7-4276-af36-7f13518c353f-kube-api-access-l5vlj\") pod \"authorino-6db5c47c98-spmzs\" (UID: \"e26fb045-02a7-4276-af36-7f13518c353f\") " pod="kuadrant-system/authorino-6db5c47c98-spmzs" Jul 16 17:00:41.728658 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:41.728628 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6db5c47c98-spmzs"] Jul 16 17:00:41.728878 ip-10-0-128-50 kubenswrapper[2571]: E0716 17:00:41.728860 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-l5vlj], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6db5c47c98-spmzs" podUID="e26fb045-02a7-4276-af36-7f13518c353f" Jul 16 17:00:41.759097 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:41.759076 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75c86975fd-nl4j4"] Jul 16 17:00:41.762344 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:41.762316 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75c86975fd-nl4j4" Jul 16 17:00:41.765779 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:41.765764 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 16 17:00:41.771933 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:41.771875 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5vlj\" (UniqueName: \"kubernetes.io/projected/e26fb045-02a7-4276-af36-7f13518c353f-kube-api-access-l5vlj\") pod \"authorino-6db5c47c98-spmzs\" (UID: \"e26fb045-02a7-4276-af36-7f13518c353f\") " pod="kuadrant-system/authorino-6db5c47c98-spmzs" Jul 16 17:00:41.773653 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:41.773636 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75c86975fd-nl4j4"] Jul 16 17:00:41.780506 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:41.780478 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5vlj\" (UniqueName: \"kubernetes.io/projected/e26fb045-02a7-4276-af36-7f13518c353f-kube-api-access-l5vlj\") pod \"authorino-6db5c47c98-spmzs\" (UID: \"e26fb045-02a7-4276-af36-7f13518c353f\") " pod="kuadrant-system/authorino-6db5c47c98-spmzs" Jul 16 17:00:41.872477 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:41.872458 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/46baa1cb-ef0e-4f3c-a94f-6baf024574ad-tls-cert\") pod \"authorino-75c86975fd-nl4j4\" (UID: \"46baa1cb-ef0e-4f3c-a94f-6baf024574ad\") " pod="kuadrant-system/authorino-75c86975fd-nl4j4" Jul 16 17:00:41.872594 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:41.872510 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psxnn\" (UniqueName: \"kubernetes.io/projected/46baa1cb-ef0e-4f3c-a94f-6baf024574ad-kube-api-access-psxnn\") pod \"authorino-75c86975fd-nl4j4\" (UID: \"46baa1cb-ef0e-4f3c-a94f-6baf024574ad\") " pod="kuadrant-system/authorino-75c86975fd-nl4j4" Jul 16 17:00:41.973327 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:41.973300 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/46baa1cb-ef0e-4f3c-a94f-6baf024574ad-tls-cert\") pod \"authorino-75c86975fd-nl4j4\" (UID: \"46baa1cb-ef0e-4f3c-a94f-6baf024574ad\") " pod="kuadrant-system/authorino-75c86975fd-nl4j4" Jul 16 17:00:41.973412 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:41.973340 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psxnn\" (UniqueName: \"kubernetes.io/projected/46baa1cb-ef0e-4f3c-a94f-6baf024574ad-kube-api-access-psxnn\") pod \"authorino-75c86975fd-nl4j4\" (UID: \"46baa1cb-ef0e-4f3c-a94f-6baf024574ad\") " pod="kuadrant-system/authorino-75c86975fd-nl4j4" Jul 16 17:00:41.975572 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:41.975551 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/46baa1cb-ef0e-4f3c-a94f-6baf024574ad-tls-cert\") pod \"authorino-75c86975fd-nl4j4\" (UID: \"46baa1cb-ef0e-4f3c-a94f-6baf024574ad\") " pod="kuadrant-system/authorino-75c86975fd-nl4j4" Jul 16 17:00:41.983488 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:41.983472 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-psxnn\" (UniqueName: \"kubernetes.io/projected/46baa1cb-ef0e-4f3c-a94f-6baf024574ad-kube-api-access-psxnn\") pod \"authorino-75c86975fd-nl4j4\" (UID: \"46baa1cb-ef0e-4f3c-a94f-6baf024574ad\") " pod="kuadrant-system/authorino-75c86975fd-nl4j4" Jul 16 17:00:42.072551 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:42.072533 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75c86975fd-nl4j4" Jul 16 17:00:42.217463 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:42.217437 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75c86975fd-nl4j4"] Jul 16 17:00:42.218875 ip-10-0-128-50 kubenswrapper[2571]: W0716 17:00:42.218851 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod46baa1cb_ef0e_4f3c_a94f_6baf024574ad.slice/crio-b300b6155df82681ea65bfbad226b8d7ead0305e29999c38e36968f02fca5ae9 WatchSource:0}: Error finding container b300b6155df82681ea65bfbad226b8d7ead0305e29999c38e36968f02fca5ae9: Status 404 returned error can't find the container with id b300b6155df82681ea65bfbad226b8d7ead0305e29999c38e36968f02fca5ae9 Jul 16 17:00:42.531369 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:42.531341 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75c86975fd-nl4j4" event={"ID":"46baa1cb-ef0e-4f3c-a94f-6baf024574ad","Type":"ContainerStarted","Data":"b300b6155df82681ea65bfbad226b8d7ead0305e29999c38e36968f02fca5ae9"} Jul 16 17:00:42.532801 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:42.532771 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-lk4ks" event={"ID":"d368b5e6-24a4-4cde-9db3-b345f6969ae6","Type":"ContainerStarted","Data":"9069e69aa41072d668690f3219dfb0ba7094e14517bdfb4ad21e7b7c73ceb35b"} Jul 16 17:00:42.532905 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:42.532810 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-lk4ks" event={"ID":"d368b5e6-24a4-4cde-9db3-b345f6969ae6","Type":"ContainerStarted","Data":"3601e1cbe066546a9d4d5f20a014f8c9714f44083cd6f362dcaf2ea01ebd3e40"} Jul 16 17:00:42.532905 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:42.532809 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-lk4ks" podUID="d368b5e6-24a4-4cde-9db3-b345f6969ae6" containerName="authorino" containerID="cri-o://9069e69aa41072d668690f3219dfb0ba7094e14517bdfb4ad21e7b7c73ceb35b" gracePeriod=30 Jul 16 17:00:42.532905 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:42.532804 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6db5c47c98-spmzs" Jul 16 17:00:42.541361 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:42.541342 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6db5c47c98-spmzs" Jul 16 17:00:42.555574 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:42.555537 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-lk4ks" podStartSLOduration=1.230126163 podStartE2EDuration="1.555519696s" totalImagesPullingTime="325.393533ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:00:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:00:42.554081578 +0000 UTC m=+594.295656876" watchObservedRunningTime="2026-07-16 17:00:42.555519696 +0000 UTC m=+594.297094989" Jul 16 17:00:42.677678 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:42.677648 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l5vlj\" (UniqueName: \"kubernetes.io/projected/e26fb045-02a7-4276-af36-7f13518c353f-kube-api-access-l5vlj\") pod \"e26fb045-02a7-4276-af36-7f13518c353f\" (UID: \"e26fb045-02a7-4276-af36-7f13518c353f\") " Jul 16 17:00:42.679600 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:42.679569 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e26fb045-02a7-4276-af36-7f13518c353f-kube-api-access-l5vlj" (OuterVolumeSpecName: "kube-api-access-l5vlj") pod "e26fb045-02a7-4276-af36-7f13518c353f" (UID: "e26fb045-02a7-4276-af36-7f13518c353f"). InnerVolumeSpecName "kube-api-access-l5vlj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:00:42.771404 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:42.771379 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-lk4ks" Jul 16 17:00:42.778428 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:42.778405 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l5vlj\" (UniqueName: \"kubernetes.io/projected/e26fb045-02a7-4276-af36-7f13518c353f-kube-api-access-l5vlj\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:00:42.879347 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:42.879326 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m9p9w\" (UniqueName: \"kubernetes.io/projected/d368b5e6-24a4-4cde-9db3-b345f6969ae6-kube-api-access-m9p9w\") pod \"d368b5e6-24a4-4cde-9db3-b345f6969ae6\" (UID: \"d368b5e6-24a4-4cde-9db3-b345f6969ae6\") " Jul 16 17:00:42.881225 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:42.881197 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d368b5e6-24a4-4cde-9db3-b345f6969ae6-kube-api-access-m9p9w" (OuterVolumeSpecName: "kube-api-access-m9p9w") pod "d368b5e6-24a4-4cde-9db3-b345f6969ae6" (UID: "d368b5e6-24a4-4cde-9db3-b345f6969ae6"). InnerVolumeSpecName "kube-api-access-m9p9w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:00:42.979809 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:42.979785 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m9p9w\" (UniqueName: \"kubernetes.io/projected/d368b5e6-24a4-4cde-9db3-b345f6969ae6-kube-api-access-m9p9w\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:00:43.539168 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:43.539144 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75c86975fd-nl4j4" event={"ID":"46baa1cb-ef0e-4f3c-a94f-6baf024574ad","Type":"ContainerStarted","Data":"9387ad7d598d6de1607ef06771a1d5c16f7606ce9fe98247e2623dfcca014126"} Jul 16 17:00:43.540456 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:43.540429 2571 generic.go:356] "Generic (PLEG): container finished" podID="d368b5e6-24a4-4cde-9db3-b345f6969ae6" containerID="9069e69aa41072d668690f3219dfb0ba7094e14517bdfb4ad21e7b7c73ceb35b" exitCode=0 Jul 16 17:00:43.540521 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:43.540459 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-lk4ks" Jul 16 17:00:43.540556 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:43.540516 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-lk4ks" event={"ID":"d368b5e6-24a4-4cde-9db3-b345f6969ae6","Type":"ContainerDied","Data":"9069e69aa41072d668690f3219dfb0ba7094e14517bdfb4ad21e7b7c73ceb35b"} Jul 16 17:00:43.540556 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:43.540545 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6db5c47c98-spmzs" Jul 16 17:00:43.540556 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:43.540553 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-lk4ks" event={"ID":"d368b5e6-24a4-4cde-9db3-b345f6969ae6","Type":"ContainerDied","Data":"3601e1cbe066546a9d4d5f20a014f8c9714f44083cd6f362dcaf2ea01ebd3e40"} Jul 16 17:00:43.540649 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:43.540575 2571 scope.go:117] "RemoveContainer" containerID="9069e69aa41072d668690f3219dfb0ba7094e14517bdfb4ad21e7b7c73ceb35b" Jul 16 17:00:43.550468 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:43.550452 2571 scope.go:117] "RemoveContainer" containerID="9069e69aa41072d668690f3219dfb0ba7094e14517bdfb4ad21e7b7c73ceb35b" Jul 16 17:00:43.550781 ip-10-0-128-50 kubenswrapper[2571]: E0716 17:00:43.550711 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9069e69aa41072d668690f3219dfb0ba7094e14517bdfb4ad21e7b7c73ceb35b\": container with ID starting with 9069e69aa41072d668690f3219dfb0ba7094e14517bdfb4ad21e7b7c73ceb35b not found: ID does not exist" containerID="9069e69aa41072d668690f3219dfb0ba7094e14517bdfb4ad21e7b7c73ceb35b" Jul 16 17:00:43.550829 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:43.550793 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9069e69aa41072d668690f3219dfb0ba7094e14517bdfb4ad21e7b7c73ceb35b"} err="failed to get container status \"9069e69aa41072d668690f3219dfb0ba7094e14517bdfb4ad21e7b7c73ceb35b\": rpc error: code = NotFound desc = could not find container \"9069e69aa41072d668690f3219dfb0ba7094e14517bdfb4ad21e7b7c73ceb35b\": container with ID starting with 9069e69aa41072d668690f3219dfb0ba7094e14517bdfb4ad21e7b7c73ceb35b not found: ID does not exist" Jul 16 17:00:43.558778 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:43.558740 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75c86975fd-nl4j4" podStartSLOduration=2.186935284 podStartE2EDuration="2.558707157s" totalImagesPullingTime="371.771873ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:00:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:00:43.558574271 +0000 UTC m=+595.300149569" watchObservedRunningTime="2026-07-16 17:00:43.558707157 +0000 UTC m=+595.300282455" Jul 16 17:00:43.576227 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:43.576205 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lk4ks"] Jul 16 17:00:43.582738 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:43.582701 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lk4ks"] Jul 16 17:00:43.618666 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:43.618642 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6db5c47c98-spmzs"] Jul 16 17:00:43.624036 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:43.624013 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6db5c47c98-spmzs"] Jul 16 17:00:44.776890 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:44.776858 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d368b5e6-24a4-4cde-9db3-b345f6969ae6" path="/var/lib/kubelet/pods/d368b5e6-24a4-4cde-9db3-b345f6969ae6/volumes" Jul 16 17:00:44.777395 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:44.777318 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e26fb045-02a7-4276-af36-7f13518c353f" path="/var/lib/kubelet/pods/e26fb045-02a7-4276-af36-7f13518c353f/volumes" Jul 16 17:00:48.711657 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:48.711626 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r8cn4_f637d961-672d-4aa2-a49a-fd021c2d4df2/console-operator/2.log" Jul 16 17:00:48.712265 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:48.712245 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r8cn4_f637d961-672d-4aa2-a49a-fd021c2d4df2/console-operator/2.log" Jul 16 17:00:48.720304 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:48.720284 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qd7bg_cfbe9f0b-b759-4064-9332-3b48c547c003/ovn-acl-logging/0.log" Jul 16 17:00:48.721246 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:00:48.721229 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qd7bg_cfbe9f0b-b759-4064-9332-3b48c547c003/ovn-acl-logging/0.log" Jul 16 17:02:00.268688 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:00.268655 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k"] Jul 16 17:02:00.269177 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:00.269010 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d368b5e6-24a4-4cde-9db3-b345f6969ae6" containerName="authorino" Jul 16 17:02:00.269177 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:00.269022 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d368b5e6-24a4-4cde-9db3-b345f6969ae6" containerName="authorino" Jul 16 17:02:00.269177 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:00.269085 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d368b5e6-24a4-4cde-9db3-b345f6969ae6" containerName="authorino" Jul 16 17:02:00.271963 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:00.271947 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k" Jul 16 17:02:00.274959 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:00.274939 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 16 17:02:00.275485 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:00.275459 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-tgg58\"" Jul 16 17:02:00.276404 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:00.276388 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 16 17:02:00.276505 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:00.276420 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 16 17:02:00.286194 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:00.286169 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k"] Jul 16 17:02:00.387938 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:00.387915 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc213918-a6b8-4a89-ac2e-d32e05fc6ae4-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k\" (UID: \"cc213918-a6b8-4a89-ac2e-d32e05fc6ae4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k" Jul 16 17:02:00.388039 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:00.387954 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc213918-a6b8-4a89-ac2e-d32e05fc6ae4-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k\" (UID: \"cc213918-a6b8-4a89-ac2e-d32e05fc6ae4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k" Jul 16 17:02:00.388039 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:00.387976 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc213918-a6b8-4a89-ac2e-d32e05fc6ae4-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k\" (UID: \"cc213918-a6b8-4a89-ac2e-d32e05fc6ae4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k" Jul 16 17:02:00.388111 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:00.388045 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc213918-a6b8-4a89-ac2e-d32e05fc6ae4-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k\" (UID: \"cc213918-a6b8-4a89-ac2e-d32e05fc6ae4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k" Jul 16 17:02:00.388111 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:00.388079 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sccc\" (UniqueName: \"kubernetes.io/projected/cc213918-a6b8-4a89-ac2e-d32e05fc6ae4-kube-api-access-8sccc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k\" (UID: \"cc213918-a6b8-4a89-ac2e-d32e05fc6ae4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k" Jul 16 17:02:00.388175 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:00.388140 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc213918-a6b8-4a89-ac2e-d32e05fc6ae4-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k\" (UID: \"cc213918-a6b8-4a89-ac2e-d32e05fc6ae4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k" Jul 16 17:02:00.489240 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:00.489216 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc213918-a6b8-4a89-ac2e-d32e05fc6ae4-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k\" (UID: \"cc213918-a6b8-4a89-ac2e-d32e05fc6ae4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k" Jul 16 17:02:00.489345 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:00.489250 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc213918-a6b8-4a89-ac2e-d32e05fc6ae4-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k\" (UID: \"cc213918-a6b8-4a89-ac2e-d32e05fc6ae4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k" Jul 16 17:02:00.489345 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:00.489280 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc213918-a6b8-4a89-ac2e-d32e05fc6ae4-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k\" (UID: \"cc213918-a6b8-4a89-ac2e-d32e05fc6ae4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k" Jul 16 17:02:00.489345 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:00.489310 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sccc\" (UniqueName: \"kubernetes.io/projected/cc213918-a6b8-4a89-ac2e-d32e05fc6ae4-kube-api-access-8sccc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k\" (UID: \"cc213918-a6b8-4a89-ac2e-d32e05fc6ae4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k" Jul 16 17:02:00.489460 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:00.489351 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc213918-a6b8-4a89-ac2e-d32e05fc6ae4-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k\" (UID: \"cc213918-a6b8-4a89-ac2e-d32e05fc6ae4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k" Jul 16 17:02:00.489460 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:00.489398 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc213918-a6b8-4a89-ac2e-d32e05fc6ae4-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k\" (UID: \"cc213918-a6b8-4a89-ac2e-d32e05fc6ae4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k" Jul 16 17:02:00.489670 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:00.489648 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc213918-a6b8-4a89-ac2e-d32e05fc6ae4-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k\" (UID: \"cc213918-a6b8-4a89-ac2e-d32e05fc6ae4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k" Jul 16 17:02:00.489782 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:00.489720 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc213918-a6b8-4a89-ac2e-d32e05fc6ae4-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k\" (UID: \"cc213918-a6b8-4a89-ac2e-d32e05fc6ae4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k" Jul 16 17:02:00.489870 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:00.489851 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc213918-a6b8-4a89-ac2e-d32e05fc6ae4-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k\" (UID: \"cc213918-a6b8-4a89-ac2e-d32e05fc6ae4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k" Jul 16 17:02:00.491479 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:00.491463 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc213918-a6b8-4a89-ac2e-d32e05fc6ae4-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k\" (UID: \"cc213918-a6b8-4a89-ac2e-d32e05fc6ae4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k" Jul 16 17:02:00.491762 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:00.491745 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc213918-a6b8-4a89-ac2e-d32e05fc6ae4-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k\" (UID: \"cc213918-a6b8-4a89-ac2e-d32e05fc6ae4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k" Jul 16 17:02:00.501982 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:00.501957 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sccc\" (UniqueName: \"kubernetes.io/projected/cc213918-a6b8-4a89-ac2e-d32e05fc6ae4-kube-api-access-8sccc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k\" (UID: \"cc213918-a6b8-4a89-ac2e-d32e05fc6ae4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k" Jul 16 17:02:00.581251 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:00.581195 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k" Jul 16 17:02:00.723520 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:00.723491 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k"] Jul 16 17:02:00.725281 ip-10-0-128-50 kubenswrapper[2571]: W0716 17:02:00.725253 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc213918_a6b8_4a89_ac2e_d32e05fc6ae4.slice/crio-cdaa7ee14d273bb8715e42c182441e7ce76afcbe4c0a74ebf0c0215622b5b4a3 WatchSource:0}: Error finding container cdaa7ee14d273bb8715e42c182441e7ce76afcbe4c0a74ebf0c0215622b5b4a3: Status 404 returned error can't find the container with id cdaa7ee14d273bb8715e42c182441e7ce76afcbe4c0a74ebf0c0215622b5b4a3 Jul 16 17:02:00.726529 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:00.726512 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 17:02:00.943154 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:00.943126 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k" event={"ID":"cc213918-a6b8-4a89-ac2e-d32e05fc6ae4","Type":"ContainerStarted","Data":"cdaa7ee14d273bb8715e42c182441e7ce76afcbe4c0a74ebf0c0215622b5b4a3"} Jul 16 17:02:01.950598 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:01.950569 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k" event={"ID":"cc213918-a6b8-4a89-ac2e-d32e05fc6ae4","Type":"ContainerStarted","Data":"30c060dca67ef366c2375bcc27d301ef39212f8d4e9eafc03240b53ea4271578"} Jul 16 17:02:02.003396 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:02.003343 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k" podStartSLOduration=0.974627589 podStartE2EDuration="2.00332805s" totalImagesPullingTime="1.028700461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:02:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:02:02.002229594 +0000 UTC m=+673.743804890" watchObservedRunningTime="2026-07-16 17:02:02.00332805 +0000 UTC m=+673.744903346" Jul 16 17:02:10.581581 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:10.581543 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k" Jul 16 17:02:10.582070 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:10.581615 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k" Jul 16 17:02:10.596406 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:10.596382 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k" Jul 16 17:02:11.010331 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:11.010297 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k" Jul 16 17:02:19.195761 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:19.195713 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6"] Jul 16 17:02:19.198918 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:19.198900 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6" Jul 16 17:02:19.202524 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:19.202505 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 16 17:02:19.213034 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:19.213014 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6"] Jul 16 17:02:19.227404 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:19.227374 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0253c5d2-d675-4b8c-b123-9f5da2a8ae3b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6\" (UID: \"0253c5d2-d675-4b8c-b123-9f5da2a8ae3b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6" Jul 16 17:02:19.227404 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:19.227419 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0253c5d2-d675-4b8c-b123-9f5da2a8ae3b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6\" (UID: \"0253c5d2-d675-4b8c-b123-9f5da2a8ae3b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6" Jul 16 17:02:19.227609 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:19.227457 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0253c5d2-d675-4b8c-b123-9f5da2a8ae3b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6\" (UID: \"0253c5d2-d675-4b8c-b123-9f5da2a8ae3b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6" Jul 16 17:02:19.227609 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:19.227496 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0253c5d2-d675-4b8c-b123-9f5da2a8ae3b-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6\" (UID: \"0253c5d2-d675-4b8c-b123-9f5da2a8ae3b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6" Jul 16 17:02:19.227609 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:19.227530 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0253c5d2-d675-4b8c-b123-9f5da2a8ae3b-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6\" (UID: \"0253c5d2-d675-4b8c-b123-9f5da2a8ae3b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6" Jul 16 17:02:19.227609 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:19.227557 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kbz2\" (UniqueName: \"kubernetes.io/projected/0253c5d2-d675-4b8c-b123-9f5da2a8ae3b-kube-api-access-4kbz2\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6\" (UID: \"0253c5d2-d675-4b8c-b123-9f5da2a8ae3b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6" Jul 16 17:02:19.328553 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:19.328525 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0253c5d2-d675-4b8c-b123-9f5da2a8ae3b-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6\" (UID: \"0253c5d2-d675-4b8c-b123-9f5da2a8ae3b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6" Jul 16 17:02:19.328678 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:19.328557 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0253c5d2-d675-4b8c-b123-9f5da2a8ae3b-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6\" (UID: \"0253c5d2-d675-4b8c-b123-9f5da2a8ae3b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6" Jul 16 17:02:19.328678 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:19.328575 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kbz2\" (UniqueName: \"kubernetes.io/projected/0253c5d2-d675-4b8c-b123-9f5da2a8ae3b-kube-api-access-4kbz2\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6\" (UID: \"0253c5d2-d675-4b8c-b123-9f5da2a8ae3b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6" Jul 16 17:02:19.328804 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:19.328737 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0253c5d2-d675-4b8c-b123-9f5da2a8ae3b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6\" (UID: \"0253c5d2-d675-4b8c-b123-9f5da2a8ae3b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6" Jul 16 17:02:19.328804 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:19.328783 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0253c5d2-d675-4b8c-b123-9f5da2a8ae3b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6\" (UID: \"0253c5d2-d675-4b8c-b123-9f5da2a8ae3b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6" Jul 16 17:02:19.328891 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:19.328824 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0253c5d2-d675-4b8c-b123-9f5da2a8ae3b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6\" (UID: \"0253c5d2-d675-4b8c-b123-9f5da2a8ae3b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6" Jul 16 17:02:19.329140 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:19.329110 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0253c5d2-d675-4b8c-b123-9f5da2a8ae3b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6\" (UID: \"0253c5d2-d675-4b8c-b123-9f5da2a8ae3b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6" Jul 16 17:02:19.329305 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:19.329108 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0253c5d2-d675-4b8c-b123-9f5da2a8ae3b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6\" (UID: \"0253c5d2-d675-4b8c-b123-9f5da2a8ae3b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6" Jul 16 17:02:19.329305 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:19.329284 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0253c5d2-d675-4b8c-b123-9f5da2a8ae3b-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6\" (UID: \"0253c5d2-d675-4b8c-b123-9f5da2a8ae3b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6" Jul 16 17:02:19.330837 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:19.330817 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0253c5d2-d675-4b8c-b123-9f5da2a8ae3b-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6\" (UID: \"0253c5d2-d675-4b8c-b123-9f5da2a8ae3b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6" Jul 16 17:02:19.331130 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:19.331111 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0253c5d2-d675-4b8c-b123-9f5da2a8ae3b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6\" (UID: \"0253c5d2-d675-4b8c-b123-9f5da2a8ae3b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6" Jul 16 17:02:19.338929 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:19.338909 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kbz2\" (UniqueName: \"kubernetes.io/projected/0253c5d2-d675-4b8c-b123-9f5da2a8ae3b-kube-api-access-4kbz2\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6\" (UID: \"0253c5d2-d675-4b8c-b123-9f5da2a8ae3b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6" Jul 16 17:02:19.509092 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:19.509029 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6" Jul 16 17:02:19.644008 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:19.643977 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6"] Jul 16 17:02:19.646290 ip-10-0-128-50 kubenswrapper[2571]: W0716 17:02:19.646260 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0253c5d2_d675_4b8c_b123_9f5da2a8ae3b.slice/crio-1558d712528ceed68c62d63cf36ae6b5b53118fabd2acd6d0bcbe6ea90b18bda WatchSource:0}: Error finding container 1558d712528ceed68c62d63cf36ae6b5b53118fabd2acd6d0bcbe6ea90b18bda: Status 404 returned error can't find the container with id 1558d712528ceed68c62d63cf36ae6b5b53118fabd2acd6d0bcbe6ea90b18bda Jul 16 17:02:20.047802 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:20.047775 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6" event={"ID":"0253c5d2-d675-4b8c-b123-9f5da2a8ae3b","Type":"ContainerStarted","Data":"2497f50afb6f2165e75c51d153df3058cd413d9ad93cbad258ea7f1593d24104"} Jul 16 17:02:20.047944 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:20.047816 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6" event={"ID":"0253c5d2-d675-4b8c-b123-9f5da2a8ae3b","Type":"ContainerStarted","Data":"1558d712528ceed68c62d63cf36ae6b5b53118fabd2acd6d0bcbe6ea90b18bda"} Jul 16 17:02:29.509406 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:29.509371 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6" Jul 16 17:02:29.509888 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:29.509418 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6" Jul 16 17:02:29.521927 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:29.521905 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6" Jul 16 17:02:29.545798 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:29.545753 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6" podStartSLOduration=10.25036275 podStartE2EDuration="10.545740691s" totalImagesPullingTime="295.377941ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:02:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:02:20.070266998 +0000 UTC m=+691.811842292" watchObservedRunningTime="2026-07-16 17:02:29.545740691 +0000 UTC m=+701.287315979" Jul 16 17:02:30.113917 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:02:30.113883 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6" Jul 16 17:03:40.148024 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:03:40.147992 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-657df94667-8qzkj"] Jul 16 17:03:40.151489 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:03:40.151470 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-657df94667-8qzkj" Jul 16 17:03:40.160459 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:03:40.160433 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-657df94667-8qzkj"] Jul 16 17:03:40.231809 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:03:40.231784 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7332c6ef-7c0f-4f06-9e27-f526a138a28b-tls-cert\") pod \"authorino-657df94667-8qzkj\" (UID: \"7332c6ef-7c0f-4f06-9e27-f526a138a28b\") " pod="kuadrant-system/authorino-657df94667-8qzkj" Jul 16 17:03:40.231916 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:03:40.231817 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vf2dc\" (UniqueName: \"kubernetes.io/projected/7332c6ef-7c0f-4f06-9e27-f526a138a28b-kube-api-access-vf2dc\") pod \"authorino-657df94667-8qzkj\" (UID: \"7332c6ef-7c0f-4f06-9e27-f526a138a28b\") " pod="kuadrant-system/authorino-657df94667-8qzkj" Jul 16 17:03:40.332068 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:03:40.332046 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7332c6ef-7c0f-4f06-9e27-f526a138a28b-tls-cert\") pod \"authorino-657df94667-8qzkj\" (UID: \"7332c6ef-7c0f-4f06-9e27-f526a138a28b\") " pod="kuadrant-system/authorino-657df94667-8qzkj" Jul 16 17:03:40.332172 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:03:40.332077 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vf2dc\" (UniqueName: \"kubernetes.io/projected/7332c6ef-7c0f-4f06-9e27-f526a138a28b-kube-api-access-vf2dc\") pod \"authorino-657df94667-8qzkj\" (UID: \"7332c6ef-7c0f-4f06-9e27-f526a138a28b\") " pod="kuadrant-system/authorino-657df94667-8qzkj" Jul 16 17:03:40.334447 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:03:40.334424 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7332c6ef-7c0f-4f06-9e27-f526a138a28b-tls-cert\") pod \"authorino-657df94667-8qzkj\" (UID: \"7332c6ef-7c0f-4f06-9e27-f526a138a28b\") " pod="kuadrant-system/authorino-657df94667-8qzkj" Jul 16 17:03:40.340613 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:03:40.340588 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vf2dc\" (UniqueName: \"kubernetes.io/projected/7332c6ef-7c0f-4f06-9e27-f526a138a28b-kube-api-access-vf2dc\") pod \"authorino-657df94667-8qzkj\" (UID: \"7332c6ef-7c0f-4f06-9e27-f526a138a28b\") " pod="kuadrant-system/authorino-657df94667-8qzkj" Jul 16 17:03:40.461629 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:03:40.461605 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-657df94667-8qzkj" Jul 16 17:03:40.598019 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:03:40.597993 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-657df94667-8qzkj"] Jul 16 17:03:40.599911 ip-10-0-128-50 kubenswrapper[2571]: W0716 17:03:40.599883 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7332c6ef_7c0f_4f06_9e27_f526a138a28b.slice/crio-5ddf00c5626582737fa22f3d2e2daab04c4f27320ad131da8da501471e78f5d9 WatchSource:0}: Error finding container 5ddf00c5626582737fa22f3d2e2daab04c4f27320ad131da8da501471e78f5d9: Status 404 returned error can't find the container with id 5ddf00c5626582737fa22f3d2e2daab04c4f27320ad131da8da501471e78f5d9 Jul 16 17:03:41.475062 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:03:41.475024 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-657df94667-8qzkj" event={"ID":"7332c6ef-7c0f-4f06-9e27-f526a138a28b","Type":"ContainerStarted","Data":"2baf027b20c216f0892145db5efe530b15e2dfccce9ae720c034eb4f8a93794e"} Jul 16 17:03:41.475456 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:03:41.475075 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-657df94667-8qzkj" event={"ID":"7332c6ef-7c0f-4f06-9e27-f526a138a28b","Type":"ContainerStarted","Data":"5ddf00c5626582737fa22f3d2e2daab04c4f27320ad131da8da501471e78f5d9"} Jul 16 17:03:41.494613 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:03:41.494557 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-657df94667-8qzkj" podStartSLOduration=1.092437304 podStartE2EDuration="1.49453836s" totalImagesPullingTime="402.101056ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:03:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:03:41.493873099 +0000 UTC m=+773.235448394" watchObservedRunningTime="2026-07-16 17:03:41.49453836 +0000 UTC m=+773.236113655" Jul 16 17:03:41.538322 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:03:41.538280 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75c86975fd-nl4j4"] Jul 16 17:03:41.538541 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:03:41.538512 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75c86975fd-nl4j4" podUID="46baa1cb-ef0e-4f3c-a94f-6baf024574ad" containerName="authorino" containerID="cri-o://9387ad7d598d6de1607ef06771a1d5c16f7606ce9fe98247e2623dfcca014126" gracePeriod=30 Jul 16 17:03:41.799271 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:03:41.799250 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75c86975fd-nl4j4" Jul 16 17:03:41.842057 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:03:41.841889 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/46baa1cb-ef0e-4f3c-a94f-6baf024574ad-tls-cert\") pod \"46baa1cb-ef0e-4f3c-a94f-6baf024574ad\" (UID: \"46baa1cb-ef0e-4f3c-a94f-6baf024574ad\") " Jul 16 17:03:41.842057 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:03:41.841936 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-psxnn\" (UniqueName: \"kubernetes.io/projected/46baa1cb-ef0e-4f3c-a94f-6baf024574ad-kube-api-access-psxnn\") pod \"46baa1cb-ef0e-4f3c-a94f-6baf024574ad\" (UID: \"46baa1cb-ef0e-4f3c-a94f-6baf024574ad\") " Jul 16 17:03:41.844806 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:03:41.844779 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46baa1cb-ef0e-4f3c-a94f-6baf024574ad-kube-api-access-psxnn" (OuterVolumeSpecName: "kube-api-access-psxnn") pod "46baa1cb-ef0e-4f3c-a94f-6baf024574ad" (UID: "46baa1cb-ef0e-4f3c-a94f-6baf024574ad"). InnerVolumeSpecName "kube-api-access-psxnn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:03:41.853234 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:03:41.853208 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46baa1cb-ef0e-4f3c-a94f-6baf024574ad-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "46baa1cb-ef0e-4f3c-a94f-6baf024574ad" (UID: "46baa1cb-ef0e-4f3c-a94f-6baf024574ad"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:03:41.942946 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:03:41.942904 2571 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/46baa1cb-ef0e-4f3c-a94f-6baf024574ad-tls-cert\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:03:41.942946 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:03:41.942942 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-psxnn\" (UniqueName: \"kubernetes.io/projected/46baa1cb-ef0e-4f3c-a94f-6baf024574ad-kube-api-access-psxnn\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:03:42.482703 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:03:42.482668 2571 generic.go:356] "Generic (PLEG): container finished" podID="46baa1cb-ef0e-4f3c-a94f-6baf024574ad" containerID="9387ad7d598d6de1607ef06771a1d5c16f7606ce9fe98247e2623dfcca014126" exitCode=0 Jul 16 17:03:42.483141 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:03:42.482711 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75c86975fd-nl4j4" Jul 16 17:03:42.483141 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:03:42.482753 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75c86975fd-nl4j4" event={"ID":"46baa1cb-ef0e-4f3c-a94f-6baf024574ad","Type":"ContainerDied","Data":"9387ad7d598d6de1607ef06771a1d5c16f7606ce9fe98247e2623dfcca014126"} Jul 16 17:03:42.483141 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:03:42.482794 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75c86975fd-nl4j4" event={"ID":"46baa1cb-ef0e-4f3c-a94f-6baf024574ad","Type":"ContainerDied","Data":"b300b6155df82681ea65bfbad226b8d7ead0305e29999c38e36968f02fca5ae9"} Jul 16 17:03:42.483141 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:03:42.482813 2571 scope.go:117] "RemoveContainer" containerID="9387ad7d598d6de1607ef06771a1d5c16f7606ce9fe98247e2623dfcca014126" Jul 16 17:03:42.491682 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:03:42.491667 2571 scope.go:117] "RemoveContainer" containerID="9387ad7d598d6de1607ef06771a1d5c16f7606ce9fe98247e2623dfcca014126" Jul 16 17:03:42.491917 ip-10-0-128-50 kubenswrapper[2571]: E0716 17:03:42.491899 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9387ad7d598d6de1607ef06771a1d5c16f7606ce9fe98247e2623dfcca014126\": container with ID starting with 9387ad7d598d6de1607ef06771a1d5c16f7606ce9fe98247e2623dfcca014126 not found: ID does not exist" containerID="9387ad7d598d6de1607ef06771a1d5c16f7606ce9fe98247e2623dfcca014126" Jul 16 17:03:42.491970 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:03:42.491929 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9387ad7d598d6de1607ef06771a1d5c16f7606ce9fe98247e2623dfcca014126"} err="failed to get container status \"9387ad7d598d6de1607ef06771a1d5c16f7606ce9fe98247e2623dfcca014126\": rpc error: code = NotFound desc = could not find container \"9387ad7d598d6de1607ef06771a1d5c16f7606ce9fe98247e2623dfcca014126\": container with ID starting with 9387ad7d598d6de1607ef06771a1d5c16f7606ce9fe98247e2623dfcca014126 not found: ID does not exist" Jul 16 17:03:42.507360 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:03:42.507333 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75c86975fd-nl4j4"] Jul 16 17:03:42.510492 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:03:42.510469 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75c86975fd-nl4j4"] Jul 16 17:03:42.778048 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:03:42.778016 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46baa1cb-ef0e-4f3c-a94f-6baf024574ad" path="/var/lib/kubelet/pods/46baa1cb-ef0e-4f3c-a94f-6baf024574ad/volumes" Jul 16 17:05:48.748820 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:05:48.748743 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r8cn4_f637d961-672d-4aa2-a49a-fd021c2d4df2/console-operator/2.log" Jul 16 17:05:48.749300 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:05:48.748954 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r8cn4_f637d961-672d-4aa2-a49a-fd021c2d4df2/console-operator/2.log" Jul 16 17:05:48.763653 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:05:48.763631 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qd7bg_cfbe9f0b-b759-4064-9332-3b48c547c003/ovn-acl-logging/0.log" Jul 16 17:05:48.763875 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:05:48.763855 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qd7bg_cfbe9f0b-b759-4064-9332-3b48c547c003/ovn-acl-logging/0.log" Jul 16 17:09:54.731800 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:09:54.731753 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfw8r"] Jul 16 17:09:54.732465 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:09:54.731937 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfw8r" podUID="ce787c88-f7cc-452f-9989-3199cb2353ee" containerName="manager" containerID="cri-o://bc3fdc6b32447cd9e8fbdae01dca5d41a3571c6c7e86bd3e8294d88b25e0c9db" gracePeriod=10 Jul 16 17:09:57.472538 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:09:57.472510 2571 generic.go:356] "Generic (PLEG): container finished" podID="ce787c88-f7cc-452f-9989-3199cb2353ee" containerID="bc3fdc6b32447cd9e8fbdae01dca5d41a3571c6c7e86bd3e8294d88b25e0c9db" exitCode=0 Jul 16 17:09:57.472850 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:09:57.472562 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfw8r" event={"ID":"ce787c88-f7cc-452f-9989-3199cb2353ee","Type":"ContainerDied","Data":"bc3fdc6b32447cd9e8fbdae01dca5d41a3571c6c7e86bd3e8294d88b25e0c9db"} Jul 16 17:09:57.491427 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:09:57.491405 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfw8r" Jul 16 17:09:57.576780 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:09:57.576747 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ce787c88-f7cc-452f-9989-3199cb2353ee-extensions-socket-volume\") pod \"ce787c88-f7cc-452f-9989-3199cb2353ee\" (UID: \"ce787c88-f7cc-452f-9989-3199cb2353ee\") " Jul 16 17:09:57.576905 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:09:57.576832 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mpql5\" (UniqueName: \"kubernetes.io/projected/ce787c88-f7cc-452f-9989-3199cb2353ee-kube-api-access-mpql5\") pod \"ce787c88-f7cc-452f-9989-3199cb2353ee\" (UID: \"ce787c88-f7cc-452f-9989-3199cb2353ee\") " Jul 16 17:09:57.577109 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:09:57.577088 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce787c88-f7cc-452f-9989-3199cb2353ee-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ce787c88-f7cc-452f-9989-3199cb2353ee" (UID: "ce787c88-f7cc-452f-9989-3199cb2353ee"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:09:57.578766 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:09:57.578740 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce787c88-f7cc-452f-9989-3199cb2353ee-kube-api-access-mpql5" (OuterVolumeSpecName: "kube-api-access-mpql5") pod "ce787c88-f7cc-452f-9989-3199cb2353ee" (UID: "ce787c88-f7cc-452f-9989-3199cb2353ee"). InnerVolumeSpecName "kube-api-access-mpql5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:09:57.677535 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:09:57.677503 2571 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ce787c88-f7cc-452f-9989-3199cb2353ee-extensions-socket-volume\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:09:57.677535 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:09:57.677536 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mpql5\" (UniqueName: \"kubernetes.io/projected/ce787c88-f7cc-452f-9989-3199cb2353ee-kube-api-access-mpql5\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:09:58.478695 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:09:58.478671 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfw8r" Jul 16 17:09:58.479126 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:09:58.478670 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfw8r" event={"ID":"ce787c88-f7cc-452f-9989-3199cb2353ee","Type":"ContainerDied","Data":"e70c6d93aa4b83f8a54cb40c032c3f6d3326e41d50dbe88bdb40437eef9c5cbf"} Jul 16 17:09:58.479126 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:09:58.478823 2571 scope.go:117] "RemoveContainer" containerID="bc3fdc6b32447cd9e8fbdae01dca5d41a3571c6c7e86bd3e8294d88b25e0c9db" Jul 16 17:09:58.505851 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:09:58.505824 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfw8r"] Jul 16 17:09:58.510669 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:09:58.510644 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfw8r"] Jul 16 17:09:58.777326 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:09:58.777263 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce787c88-f7cc-452f-9989-3199cb2353ee" path="/var/lib/kubelet/pods/ce787c88-f7cc-452f-9989-3199cb2353ee/volumes" Jul 16 17:10:48.798117 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:10:48.798091 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r8cn4_f637d961-672d-4aa2-a49a-fd021c2d4df2/console-operator/2.log" Jul 16 17:10:48.800558 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:10:48.800537 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r8cn4_f637d961-672d-4aa2-a49a-fd021c2d4df2/console-operator/2.log" Jul 16 17:10:48.807186 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:10:48.807164 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qd7bg_cfbe9f0b-b759-4064-9332-3b48c547c003/ovn-acl-logging/0.log" Jul 16 17:10:48.809328 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:10:48.809309 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qd7bg_cfbe9f0b-b759-4064-9332-3b48c547c003/ovn-acl-logging/0.log" Jul 16 17:10:56.032383 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:10:56.032350 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42p8f"] Jul 16 17:10:56.032784 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:10:56.032665 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46baa1cb-ef0e-4f3c-a94f-6baf024574ad" containerName="authorino" Jul 16 17:10:56.032784 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:10:56.032675 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="46baa1cb-ef0e-4f3c-a94f-6baf024574ad" containerName="authorino" Jul 16 17:10:56.032784 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:10:56.032684 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce787c88-f7cc-452f-9989-3199cb2353ee" containerName="manager" Jul 16 17:10:56.032784 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:10:56.032689 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce787c88-f7cc-452f-9989-3199cb2353ee" containerName="manager" Jul 16 17:10:56.032784 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:10:56.032775 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="46baa1cb-ef0e-4f3c-a94f-6baf024574ad" containerName="authorino" Jul 16 17:10:56.032784 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:10:56.032787 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce787c88-f7cc-452f-9989-3199cb2353ee" containerName="manager" Jul 16 17:10:56.036400 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:10:56.036382 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42p8f" Jul 16 17:10:56.039349 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:10:56.039319 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-tjtrx\"" Jul 16 17:10:56.057075 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:10:56.057047 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42p8f"] Jul 16 17:10:56.120629 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:10:56.120601 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kv28\" (UniqueName: \"kubernetes.io/projected/238dde27-77a3-45a2-9ba4-ccc72f0bf58b-kube-api-access-2kv28\") pod \"kuadrant-operator-controller-manager-86878c769f-42p8f\" (UID: \"238dde27-77a3-45a2-9ba4-ccc72f0bf58b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42p8f" Jul 16 17:10:56.120776 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:10:56.120640 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/238dde27-77a3-45a2-9ba4-ccc72f0bf58b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-42p8f\" (UID: \"238dde27-77a3-45a2-9ba4-ccc72f0bf58b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42p8f" Jul 16 17:10:56.221702 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:10:56.221676 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kv28\" (UniqueName: \"kubernetes.io/projected/238dde27-77a3-45a2-9ba4-ccc72f0bf58b-kube-api-access-2kv28\") pod \"kuadrant-operator-controller-manager-86878c769f-42p8f\" (UID: \"238dde27-77a3-45a2-9ba4-ccc72f0bf58b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42p8f" Jul 16 17:10:56.221842 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:10:56.221714 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/238dde27-77a3-45a2-9ba4-ccc72f0bf58b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-42p8f\" (UID: \"238dde27-77a3-45a2-9ba4-ccc72f0bf58b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42p8f" Jul 16 17:10:56.222041 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:10:56.222024 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/238dde27-77a3-45a2-9ba4-ccc72f0bf58b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-42p8f\" (UID: \"238dde27-77a3-45a2-9ba4-ccc72f0bf58b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42p8f" Jul 16 17:10:56.234096 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:10:56.234070 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kv28\" (UniqueName: \"kubernetes.io/projected/238dde27-77a3-45a2-9ba4-ccc72f0bf58b-kube-api-access-2kv28\") pod \"kuadrant-operator-controller-manager-86878c769f-42p8f\" (UID: \"238dde27-77a3-45a2-9ba4-ccc72f0bf58b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42p8f" Jul 16 17:10:56.347341 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:10:56.347283 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42p8f" Jul 16 17:10:56.489560 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:10:56.489536 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42p8f"] Jul 16 17:10:56.490916 ip-10-0-128-50 kubenswrapper[2571]: W0716 17:10:56.490882 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod238dde27_77a3_45a2_9ba4_ccc72f0bf58b.slice/crio-fde024e077653a32420ad3eb66de1a2a15c0eed6e4d9b621dba34a511fdca7bd WatchSource:0}: Error finding container fde024e077653a32420ad3eb66de1a2a15c0eed6e4d9b621dba34a511fdca7bd: Status 404 returned error can't find the container with id fde024e077653a32420ad3eb66de1a2a15c0eed6e4d9b621dba34a511fdca7bd Jul 16 17:10:56.800636 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:10:56.800605 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42p8f" event={"ID":"238dde27-77a3-45a2-9ba4-ccc72f0bf58b","Type":"ContainerStarted","Data":"86c8b3937b94c8ea6cd8ca264b542a1502723488ff712a800e81a7fe46052c62"} Jul 16 17:10:56.800805 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:10:56.800753 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42p8f" Jul 16 17:10:56.800805 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:10:56.800797 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42p8f" event={"ID":"238dde27-77a3-45a2-9ba4-ccc72f0bf58b","Type":"ContainerStarted","Data":"fde024e077653a32420ad3eb66de1a2a15c0eed6e4d9b621dba34a511fdca7bd"} Jul 16 17:10:56.830266 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:10:56.830223 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42p8f" podStartSLOduration=0.830208719 podStartE2EDuration="830.208719ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:10:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:10:56.829283353 +0000 UTC m=+1208.570858670" watchObservedRunningTime="2026-07-16 17:10:56.830208719 +0000 UTC m=+1208.571784017" Jul 16 17:11:07.808079 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:11:07.808051 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42p8f" Jul 16 17:15:48.836291 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:15:48.836263 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r8cn4_f637d961-672d-4aa2-a49a-fd021c2d4df2/console-operator/2.log" Jul 16 17:15:48.838521 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:15:48.838496 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r8cn4_f637d961-672d-4aa2-a49a-fd021c2d4df2/console-operator/2.log" Jul 16 17:15:48.844949 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:15:48.844927 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qd7bg_cfbe9f0b-b759-4064-9332-3b48c547c003/ovn-acl-logging/0.log" Jul 16 17:15:48.846861 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:15:48.846843 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qd7bg_cfbe9f0b-b759-4064-9332-3b48c547c003/ovn-acl-logging/0.log" Jul 16 17:17:29.788987 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:29.788958 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp"] Jul 16 17:17:29.792448 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:29.792427 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" Jul 16 17:17:29.795606 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:29.795582 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 16 17:17:29.795764 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:29.795582 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-ec0988f9-openshift-default-dockercfg-44489\"" Jul 16 17:17:29.820494 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:29.820469 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp"] Jul 16 17:17:29.829274 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:29.829250 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f15e9f4e-968e-468e-98be-d124ddc7d8b4-istio-token\") pod \"e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp\" (UID: \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\") " pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" Jul 16 17:17:29.829366 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:29.829315 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f15e9f4e-968e-468e-98be-d124ddc7d8b4-istio-envoy\") pod \"e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp\" (UID: \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\") " pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" Jul 16 17:17:29.829405 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:29.829367 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f15e9f4e-968e-468e-98be-d124ddc7d8b4-workload-socket\") pod \"e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp\" (UID: \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\") " pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" Jul 16 17:17:29.829405 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:29.829392 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmdnk\" (UniqueName: \"kubernetes.io/projected/f15e9f4e-968e-468e-98be-d124ddc7d8b4-kube-api-access-bmdnk\") pod \"e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp\" (UID: \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\") " pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" Jul 16 17:17:29.829470 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:29.829420 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f15e9f4e-968e-468e-98be-d124ddc7d8b4-istio-data\") pod \"e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp\" (UID: \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\") " pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" Jul 16 17:17:29.829470 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:29.829453 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f15e9f4e-968e-468e-98be-d124ddc7d8b4-workload-certs\") pod \"e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp\" (UID: \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\") " pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" Jul 16 17:17:29.829534 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:29.829476 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f15e9f4e-968e-468e-98be-d124ddc7d8b4-istio-podinfo\") pod \"e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp\" (UID: \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\") " pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" Jul 16 17:17:29.829598 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:29.829574 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f15e9f4e-968e-468e-98be-d124ddc7d8b4-credential-socket\") pod \"e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp\" (UID: \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\") " pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" Jul 16 17:17:29.829646 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:29.829611 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f15e9f4e-968e-468e-98be-d124ddc7d8b4-istiod-ca-cert\") pod \"e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp\" (UID: \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\") " pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" Jul 16 17:17:29.930070 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:29.930040 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f15e9f4e-968e-468e-98be-d124ddc7d8b4-workload-certs\") pod \"e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp\" (UID: \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\") " pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" Jul 16 17:17:29.930070 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:29.930071 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f15e9f4e-968e-468e-98be-d124ddc7d8b4-istio-podinfo\") pod \"e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp\" (UID: \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\") " pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" Jul 16 17:17:29.930277 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:29.930135 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f15e9f4e-968e-468e-98be-d124ddc7d8b4-credential-socket\") pod \"e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp\" (UID: \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\") " pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" Jul 16 17:17:29.930277 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:29.930153 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f15e9f4e-968e-468e-98be-d124ddc7d8b4-istiod-ca-cert\") pod \"e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp\" (UID: \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\") " pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" Jul 16 17:17:29.930277 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:29.930174 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f15e9f4e-968e-468e-98be-d124ddc7d8b4-istio-token\") pod \"e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp\" (UID: \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\") " pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" Jul 16 17:17:29.930457 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:29.930435 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f15e9f4e-968e-468e-98be-d124ddc7d8b4-workload-certs\") pod \"e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp\" (UID: \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\") " pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" Jul 16 17:17:29.930513 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:29.930433 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f15e9f4e-968e-468e-98be-d124ddc7d8b4-istio-envoy\") pod \"e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp\" (UID: \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\") " pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" Jul 16 17:17:29.930561 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:29.930516 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f15e9f4e-968e-468e-98be-d124ddc7d8b4-credential-socket\") pod \"e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp\" (UID: \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\") " pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" Jul 16 17:17:29.930561 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:29.930538 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f15e9f4e-968e-468e-98be-d124ddc7d8b4-workload-socket\") pod \"e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp\" (UID: \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\") " pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" Jul 16 17:17:29.930663 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:29.930571 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmdnk\" (UniqueName: \"kubernetes.io/projected/f15e9f4e-968e-468e-98be-d124ddc7d8b4-kube-api-access-bmdnk\") pod \"e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp\" (UID: \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\") " pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" Jul 16 17:17:29.930663 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:29.930603 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f15e9f4e-968e-468e-98be-d124ddc7d8b4-istio-data\") pod \"e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp\" (UID: \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\") " pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" Jul 16 17:17:29.930938 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:29.930919 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f15e9f4e-968e-468e-98be-d124ddc7d8b4-workload-socket\") pod \"e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp\" (UID: \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\") " pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" Jul 16 17:17:29.931038 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:29.930957 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f15e9f4e-968e-468e-98be-d124ddc7d8b4-istio-data\") pod \"e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp\" (UID: \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\") " pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" Jul 16 17:17:29.931206 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:29.931186 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f15e9f4e-968e-468e-98be-d124ddc7d8b4-istiod-ca-cert\") pod \"e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp\" (UID: \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\") " pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" Jul 16 17:17:29.932546 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:29.932528 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f15e9f4e-968e-468e-98be-d124ddc7d8b4-istio-envoy\") pod \"e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp\" (UID: \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\") " pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" Jul 16 17:17:29.932777 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:29.932760 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f15e9f4e-968e-468e-98be-d124ddc7d8b4-istio-podinfo\") pod \"e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp\" (UID: \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\") " pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" Jul 16 17:17:29.942489 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:29.942464 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f15e9f4e-968e-468e-98be-d124ddc7d8b4-istio-token\") pod \"e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp\" (UID: \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\") " pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" Jul 16 17:17:29.942954 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:29.942934 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmdnk\" (UniqueName: \"kubernetes.io/projected/f15e9f4e-968e-468e-98be-d124ddc7d8b4-kube-api-access-bmdnk\") pod \"e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp\" (UID: \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\") " pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" Jul 16 17:17:30.102680 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:30.102602 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" Jul 16 17:17:30.242565 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:30.242533 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp"] Jul 16 17:17:30.244175 ip-10-0-128-50 kubenswrapper[2571]: W0716 17:17:30.244149 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf15e9f4e_968e_468e_98be_d124ddc7d8b4.slice/crio-b27f040f912532c71f4bef38cab93f86f5d1efd2cfe7f13a2522fc1639bfd6f5 WatchSource:0}: Error finding container b27f040f912532c71f4bef38cab93f86f5d1efd2cfe7f13a2522fc1639bfd6f5: Status 404 returned error can't find the container with id b27f040f912532c71f4bef38cab93f86f5d1efd2cfe7f13a2522fc1639bfd6f5 Jul 16 17:17:30.246263 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:30.246230 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 17:17:30.246371 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:30.246294 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 17:17:30.246371 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:30.246321 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 17:17:30.851645 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:30.851612 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" event={"ID":"f15e9f4e-968e-468e-98be-d124ddc7d8b4","Type":"ContainerStarted","Data":"19a737da1cfed1170e6eaff5e950e750d2f76a3b2566d7ddfb7f534192765e88"} Jul 16 17:17:30.852056 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:30.851666 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" event={"ID":"f15e9f4e-968e-468e-98be-d124ddc7d8b4","Type":"ContainerStarted","Data":"b27f040f912532c71f4bef38cab93f86f5d1efd2cfe7f13a2522fc1639bfd6f5"} Jul 16 17:17:30.881698 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:30.881644 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" podStartSLOduration=1.881629029 podStartE2EDuration="1.881629029s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:17:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:17:30.879830196 +0000 UTC m=+1602.621405492" watchObservedRunningTime="2026-07-16 17:17:30.881629029 +0000 UTC m=+1602.623204324" Jul 16 17:17:31.103791 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:31.103699 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" Jul 16 17:17:31.108271 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:31.108249 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" Jul 16 17:17:31.857355 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:31.857324 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" Jul 16 17:17:31.858155 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:31.858138 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" Jul 16 17:17:36.282136 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:36.282097 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp"] Jul 16 17:17:36.282554 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:36.282283 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" podUID="f15e9f4e-968e-468e-98be-d124ddc7d8b4" containerName="istio-proxy" containerID="cri-o://19a737da1cfed1170e6eaff5e950e750d2f76a3b2566d7ddfb7f534192765e88" gracePeriod=30 Jul 16 17:17:42.727796 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:42.727769 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" Jul 16 17:17:42.845859 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:42.845778 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f15e9f4e-968e-468e-98be-d124ddc7d8b4-istio-envoy\") pod \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\" (UID: \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\") " Jul 16 17:17:42.845859 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:42.845817 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f15e9f4e-968e-468e-98be-d124ddc7d8b4-istio-token\") pod \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\" (UID: \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\") " Jul 16 17:17:42.846068 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:42.845874 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f15e9f4e-968e-468e-98be-d124ddc7d8b4-credential-socket\") pod \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\" (UID: \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\") " Jul 16 17:17:42.846068 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:42.845892 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f15e9f4e-968e-468e-98be-d124ddc7d8b4-workload-certs\") pod \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\" (UID: \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\") " Jul 16 17:17:42.846068 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:42.845911 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bmdnk\" (UniqueName: \"kubernetes.io/projected/f15e9f4e-968e-468e-98be-d124ddc7d8b4-kube-api-access-bmdnk\") pod \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\" (UID: \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\") " Jul 16 17:17:42.846068 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:42.845938 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f15e9f4e-968e-468e-98be-d124ddc7d8b4-istiod-ca-cert\") pod \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\" (UID: \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\") " Jul 16 17:17:42.846068 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:42.845964 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f15e9f4e-968e-468e-98be-d124ddc7d8b4-workload-socket\") pod \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\" (UID: \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\") " Jul 16 17:17:42.846068 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:42.845992 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f15e9f4e-968e-468e-98be-d124ddc7d8b4-istio-podinfo\") pod \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\" (UID: \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\") " Jul 16 17:17:42.846068 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:42.846062 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f15e9f4e-968e-468e-98be-d124ddc7d8b4-istio-data\") pod \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\" (UID: \"f15e9f4e-968e-468e-98be-d124ddc7d8b4\") " Jul 16 17:17:42.846423 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:42.846107 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f15e9f4e-968e-468e-98be-d124ddc7d8b4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f15e9f4e-968e-468e-98be-d124ddc7d8b4" (UID: "f15e9f4e-968e-468e-98be-d124ddc7d8b4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:17:42.846423 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:42.846176 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f15e9f4e-968e-468e-98be-d124ddc7d8b4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f15e9f4e-968e-468e-98be-d124ddc7d8b4" (UID: "f15e9f4e-968e-468e-98be-d124ddc7d8b4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:17:42.846423 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:42.846284 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f15e9f4e-968e-468e-98be-d124ddc7d8b4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f15e9f4e-968e-468e-98be-d124ddc7d8b4" (UID: "f15e9f4e-968e-468e-98be-d124ddc7d8b4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:17:42.846423 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:42.846294 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f15e9f4e-968e-468e-98be-d124ddc7d8b4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f15e9f4e-968e-468e-98be-d124ddc7d8b4" (UID: "f15e9f4e-968e-468e-98be-d124ddc7d8b4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:17:42.846423 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:42.846408 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f15e9f4e-968e-468e-98be-d124ddc7d8b4-credential-socket\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:17:42.846659 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:42.846426 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f15e9f4e-968e-468e-98be-d124ddc7d8b4-workload-certs\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:17:42.846659 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:42.846441 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f15e9f4e-968e-468e-98be-d124ddc7d8b4-istiod-ca-cert\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:17:42.846659 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:42.846454 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f15e9f4e-968e-468e-98be-d124ddc7d8b4-workload-socket\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:17:42.847002 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:42.846977 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f15e9f4e-968e-468e-98be-d124ddc7d8b4-istio-data" (OuterVolumeSpecName: "istio-data") pod "f15e9f4e-968e-468e-98be-d124ddc7d8b4" (UID: "f15e9f4e-968e-468e-98be-d124ddc7d8b4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:17:42.848094 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:42.848072 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f15e9f4e-968e-468e-98be-d124ddc7d8b4-istio-token" (OuterVolumeSpecName: "istio-token") pod "f15e9f4e-968e-468e-98be-d124ddc7d8b4" (UID: "f15e9f4e-968e-468e-98be-d124ddc7d8b4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:17:42.848168 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:42.848092 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f15e9f4e-968e-468e-98be-d124ddc7d8b4-kube-api-access-bmdnk" (OuterVolumeSpecName: "kube-api-access-bmdnk") pod "f15e9f4e-968e-468e-98be-d124ddc7d8b4" (UID: "f15e9f4e-968e-468e-98be-d124ddc7d8b4"). InnerVolumeSpecName "kube-api-access-bmdnk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:17:42.848333 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:42.848318 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f15e9f4e-968e-468e-98be-d124ddc7d8b4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f15e9f4e-968e-468e-98be-d124ddc7d8b4" (UID: "f15e9f4e-968e-468e-98be-d124ddc7d8b4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:17:42.848382 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:42.848362 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f15e9f4e-968e-468e-98be-d124ddc7d8b4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f15e9f4e-968e-468e-98be-d124ddc7d8b4" (UID: "f15e9f4e-968e-468e-98be-d124ddc7d8b4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 17:17:42.923584 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:42.923552 2571 generic.go:356] "Generic (PLEG): container finished" podID="f15e9f4e-968e-468e-98be-d124ddc7d8b4" containerID="19a737da1cfed1170e6eaff5e950e750d2f76a3b2566d7ddfb7f534192765e88" exitCode=0 Jul 16 17:17:42.923713 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:42.923595 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" Jul 16 17:17:42.923713 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:42.923643 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" event={"ID":"f15e9f4e-968e-468e-98be-d124ddc7d8b4","Type":"ContainerDied","Data":"19a737da1cfed1170e6eaff5e950e750d2f76a3b2566d7ddfb7f534192765e88"} Jul 16 17:17:42.923713 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:42.923698 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp" event={"ID":"f15e9f4e-968e-468e-98be-d124ddc7d8b4","Type":"ContainerDied","Data":"b27f040f912532c71f4bef38cab93f86f5d1efd2cfe7f13a2522fc1639bfd6f5"} Jul 16 17:17:42.923846 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:42.923749 2571 scope.go:117] "RemoveContainer" containerID="19a737da1cfed1170e6eaff5e950e750d2f76a3b2566d7ddfb7f534192765e88" Jul 16 17:17:42.933406 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:42.933390 2571 scope.go:117] "RemoveContainer" containerID="19a737da1cfed1170e6eaff5e950e750d2f76a3b2566d7ddfb7f534192765e88" Jul 16 17:17:42.933644 ip-10-0-128-50 kubenswrapper[2571]: E0716 17:17:42.933623 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19a737da1cfed1170e6eaff5e950e750d2f76a3b2566d7ddfb7f534192765e88\": container with ID starting with 19a737da1cfed1170e6eaff5e950e750d2f76a3b2566d7ddfb7f534192765e88 not found: ID does not exist" containerID="19a737da1cfed1170e6eaff5e950e750d2f76a3b2566d7ddfb7f534192765e88" Jul 16 17:17:42.933694 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:42.933656 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19a737da1cfed1170e6eaff5e950e750d2f76a3b2566d7ddfb7f534192765e88"} err="failed to get container status \"19a737da1cfed1170e6eaff5e950e750d2f76a3b2566d7ddfb7f534192765e88\": rpc error: code = NotFound desc = could not find container \"19a737da1cfed1170e6eaff5e950e750d2f76a3b2566d7ddfb7f534192765e88\": container with ID starting with 19a737da1cfed1170e6eaff5e950e750d2f76a3b2566d7ddfb7f534192765e88 not found: ID does not exist" Jul 16 17:17:42.947016 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:42.946988 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f15e9f4e-968e-468e-98be-d124ddc7d8b4-istio-data\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:17:42.947016 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:42.947011 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f15e9f4e-968e-468e-98be-d124ddc7d8b4-istio-envoy\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:17:42.947016 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:42.947019 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f15e9f4e-968e-468e-98be-d124ddc7d8b4-istio-token\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:17:42.947166 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:42.947027 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmdnk\" (UniqueName: \"kubernetes.io/projected/f15e9f4e-968e-468e-98be-d124ddc7d8b4-kube-api-access-bmdnk\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:17:42.947166 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:42.947036 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f15e9f4e-968e-468e-98be-d124ddc7d8b4-istio-podinfo\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:17:42.948943 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:42.948924 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp"] Jul 16 17:17:42.955933 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:42.955910 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-ec0988f9-openshift-default-9d9b4d667-qsptp"] Jul 16 17:17:44.777036 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:44.777003 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f15e9f4e-968e-468e-98be-d124ddc7d8b4" path="/var/lib/kubelet/pods/f15e9f4e-968e-468e-98be-d124ddc7d8b4/volumes" Jul 16 17:17:50.667191 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:50.667155 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn"] Jul 16 17:17:50.668202 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:50.668177 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f15e9f4e-968e-468e-98be-d124ddc7d8b4" containerName="istio-proxy" Jul 16 17:17:50.668202 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:50.668204 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f15e9f4e-968e-468e-98be-d124ddc7d8b4" containerName="istio-proxy" Jul 16 17:17:50.668374 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:50.668298 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="f15e9f4e-968e-468e-98be-d124ddc7d8b4" containerName="istio-proxy" Jul 16 17:17:50.677345 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:50.677319 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" Jul 16 17:17:50.680702 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:50.680676 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 16 17:17:50.680820 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:50.680707 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-b593195d-gw-openshift-default-dockercfg-4ldkz\"" Jul 16 17:17:50.687167 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:50.687142 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn"] Jul 16 17:17:50.812059 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:50.812021 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59434dc4-0f1f-4425-bca8-3192318254d7-istiod-ca-cert\") pod \"e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn\" (UID: \"59434dc4-0f1f-4425-bca8-3192318254d7\") " pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" Jul 16 17:17:50.812059 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:50.812064 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59434dc4-0f1f-4425-bca8-3192318254d7-istio-podinfo\") pod \"e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn\" (UID: \"59434dc4-0f1f-4425-bca8-3192318254d7\") " pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" Jul 16 17:17:50.812262 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:50.812093 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59434dc4-0f1f-4425-bca8-3192318254d7-istio-data\") pod \"e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn\" (UID: \"59434dc4-0f1f-4425-bca8-3192318254d7\") " pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" Jul 16 17:17:50.812262 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:50.812155 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8s8g5\" (UniqueName: \"kubernetes.io/projected/59434dc4-0f1f-4425-bca8-3192318254d7-kube-api-access-8s8g5\") pod \"e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn\" (UID: \"59434dc4-0f1f-4425-bca8-3192318254d7\") " pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" Jul 16 17:17:50.812262 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:50.812205 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59434dc4-0f1f-4425-bca8-3192318254d7-istio-envoy\") pod \"e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn\" (UID: \"59434dc4-0f1f-4425-bca8-3192318254d7\") " pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" Jul 16 17:17:50.812373 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:50.812263 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59434dc4-0f1f-4425-bca8-3192318254d7-workload-certs\") pod \"e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn\" (UID: \"59434dc4-0f1f-4425-bca8-3192318254d7\") " pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" Jul 16 17:17:50.812373 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:50.812288 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59434dc4-0f1f-4425-bca8-3192318254d7-istio-token\") pod \"e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn\" (UID: \"59434dc4-0f1f-4425-bca8-3192318254d7\") " pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" Jul 16 17:17:50.812373 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:50.812364 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59434dc4-0f1f-4425-bca8-3192318254d7-workload-socket\") pod \"e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn\" (UID: \"59434dc4-0f1f-4425-bca8-3192318254d7\") " pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" Jul 16 17:17:50.812478 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:50.812380 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59434dc4-0f1f-4425-bca8-3192318254d7-credential-socket\") pod \"e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn\" (UID: \"59434dc4-0f1f-4425-bca8-3192318254d7\") " pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" Jul 16 17:17:50.913469 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:50.913440 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59434dc4-0f1f-4425-bca8-3192318254d7-workload-certs\") pod \"e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn\" (UID: \"59434dc4-0f1f-4425-bca8-3192318254d7\") " pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" Jul 16 17:17:50.913606 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:50.913475 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59434dc4-0f1f-4425-bca8-3192318254d7-istio-token\") pod \"e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn\" (UID: \"59434dc4-0f1f-4425-bca8-3192318254d7\") " pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" Jul 16 17:17:50.913606 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:50.913533 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59434dc4-0f1f-4425-bca8-3192318254d7-workload-socket\") pod \"e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn\" (UID: \"59434dc4-0f1f-4425-bca8-3192318254d7\") " pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" Jul 16 17:17:50.913606 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:50.913555 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59434dc4-0f1f-4425-bca8-3192318254d7-credential-socket\") pod \"e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn\" (UID: \"59434dc4-0f1f-4425-bca8-3192318254d7\") " pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" Jul 16 17:17:50.913720 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:50.913678 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59434dc4-0f1f-4425-bca8-3192318254d7-istiod-ca-cert\") pod \"e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn\" (UID: \"59434dc4-0f1f-4425-bca8-3192318254d7\") " pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" Jul 16 17:17:50.913790 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:50.913719 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59434dc4-0f1f-4425-bca8-3192318254d7-istio-podinfo\") pod \"e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn\" (UID: \"59434dc4-0f1f-4425-bca8-3192318254d7\") " pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" Jul 16 17:17:50.913790 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:50.913778 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59434dc4-0f1f-4425-bca8-3192318254d7-istio-data\") pod \"e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn\" (UID: \"59434dc4-0f1f-4425-bca8-3192318254d7\") " pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" Jul 16 17:17:50.913900 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:50.913819 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8s8g5\" (UniqueName: \"kubernetes.io/projected/59434dc4-0f1f-4425-bca8-3192318254d7-kube-api-access-8s8g5\") pod \"e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn\" (UID: \"59434dc4-0f1f-4425-bca8-3192318254d7\") " pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" Jul 16 17:17:50.913900 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:50.913845 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59434dc4-0f1f-4425-bca8-3192318254d7-istio-envoy\") pod \"e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn\" (UID: \"59434dc4-0f1f-4425-bca8-3192318254d7\") " pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" Jul 16 17:17:50.914001 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:50.913930 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59434dc4-0f1f-4425-bca8-3192318254d7-workload-socket\") pod \"e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn\" (UID: \"59434dc4-0f1f-4425-bca8-3192318254d7\") " pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" Jul 16 17:17:50.914001 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:50.913950 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59434dc4-0f1f-4425-bca8-3192318254d7-workload-certs\") pod \"e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn\" (UID: \"59434dc4-0f1f-4425-bca8-3192318254d7\") " pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" Jul 16 17:17:50.914094 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:50.914082 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59434dc4-0f1f-4425-bca8-3192318254d7-credential-socket\") pod \"e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn\" (UID: \"59434dc4-0f1f-4425-bca8-3192318254d7\") " pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" Jul 16 17:17:50.914291 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:50.914270 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59434dc4-0f1f-4425-bca8-3192318254d7-istio-data\") pod \"e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn\" (UID: \"59434dc4-0f1f-4425-bca8-3192318254d7\") " pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" Jul 16 17:17:50.914481 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:50.914463 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59434dc4-0f1f-4425-bca8-3192318254d7-istiod-ca-cert\") pod \"e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn\" (UID: \"59434dc4-0f1f-4425-bca8-3192318254d7\") " pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" Jul 16 17:17:50.916019 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:50.915999 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59434dc4-0f1f-4425-bca8-3192318254d7-istio-envoy\") pod \"e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn\" (UID: \"59434dc4-0f1f-4425-bca8-3192318254d7\") " pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" Jul 16 17:17:50.916238 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:50.916220 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59434dc4-0f1f-4425-bca8-3192318254d7-istio-podinfo\") pod \"e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn\" (UID: \"59434dc4-0f1f-4425-bca8-3192318254d7\") " pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" Jul 16 17:17:50.924236 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:50.924217 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59434dc4-0f1f-4425-bca8-3192318254d7-istio-token\") pod \"e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn\" (UID: \"59434dc4-0f1f-4425-bca8-3192318254d7\") " pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" Jul 16 17:17:50.925707 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:50.925688 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8s8g5\" (UniqueName: \"kubernetes.io/projected/59434dc4-0f1f-4425-bca8-3192318254d7-kube-api-access-8s8g5\") pod \"e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn\" (UID: \"59434dc4-0f1f-4425-bca8-3192318254d7\") " pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" Jul 16 17:17:50.990190 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:50.990168 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" Jul 16 17:17:51.137752 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:51.137698 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn"] Jul 16 17:17:51.138524 ip-10-0-128-50 kubenswrapper[2571]: W0716 17:17:51.138501 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59434dc4_0f1f_4425_bca8_3192318254d7.slice/crio-cc758b627c77fd82eb68bc01be81cf0200c9e5c2b2bf59d64a9be6f41407229c WatchSource:0}: Error finding container cc758b627c77fd82eb68bc01be81cf0200c9e5c2b2bf59d64a9be6f41407229c: Status 404 returned error can't find the container with id cc758b627c77fd82eb68bc01be81cf0200c9e5c2b2bf59d64a9be6f41407229c Jul 16 17:17:51.140529 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:51.140494 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 17:17:51.140607 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:51.140559 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 17:17:51.140607 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:51.140603 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 17:17:51.970011 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:51.969976 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" event={"ID":"59434dc4-0f1f-4425-bca8-3192318254d7","Type":"ContainerStarted","Data":"af5c5d2c6076b457f663eb267b8fcc5f3a8766f7820e26279d86e14077d1522a"} Jul 16 17:17:51.970391 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:51.970027 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" event={"ID":"59434dc4-0f1f-4425-bca8-3192318254d7","Type":"ContainerStarted","Data":"cc758b627c77fd82eb68bc01be81cf0200c9e5c2b2bf59d64a9be6f41407229c"} Jul 16 17:17:51.991039 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:51.991021 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" Jul 16 17:17:51.998701 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:51.998645 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" podStartSLOduration=1.998625782 podStartE2EDuration="1.998625782s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:17:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:17:51.996092245 +0000 UTC m=+1623.737667540" watchObservedRunningTime="2026-07-16 17:17:51.998625782 +0000 UTC m=+1623.740201079" Jul 16 17:17:52.996991 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:52.996959 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" Jul 16 17:17:53.980340 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:53.980313 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" Jul 16 17:17:53.981117 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:53.981098 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" Jul 16 17:17:56.691096 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:56.691062 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn"] Jul 16 17:17:56.998065 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:17:56.997718 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" podUID="59434dc4-0f1f-4425-bca8-3192318254d7" containerName="istio-proxy" containerID="cri-o://af5c5d2c6076b457f663eb267b8fcc5f3a8766f7820e26279d86e14077d1522a" gracePeriod=30 Jul 16 17:18:02.384874 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:02.384848 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" Jul 16 17:18:02.517505 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:02.517477 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59434dc4-0f1f-4425-bca8-3192318254d7-istio-data\") pod \"59434dc4-0f1f-4425-bca8-3192318254d7\" (UID: \"59434dc4-0f1f-4425-bca8-3192318254d7\") " Jul 16 17:18:02.517670 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:02.517526 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59434dc4-0f1f-4425-bca8-3192318254d7-credential-socket\") pod \"59434dc4-0f1f-4425-bca8-3192318254d7\" (UID: \"59434dc4-0f1f-4425-bca8-3192318254d7\") " Jul 16 17:18:02.517670 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:02.517546 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59434dc4-0f1f-4425-bca8-3192318254d7-istio-podinfo\") pod \"59434dc4-0f1f-4425-bca8-3192318254d7\" (UID: \"59434dc4-0f1f-4425-bca8-3192318254d7\") " Jul 16 17:18:02.517670 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:02.517562 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59434dc4-0f1f-4425-bca8-3192318254d7-workload-certs\") pod \"59434dc4-0f1f-4425-bca8-3192318254d7\" (UID: \"59434dc4-0f1f-4425-bca8-3192318254d7\") " Jul 16 17:18:02.517670 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:02.517592 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59434dc4-0f1f-4425-bca8-3192318254d7-istiod-ca-cert\") pod \"59434dc4-0f1f-4425-bca8-3192318254d7\" (UID: \"59434dc4-0f1f-4425-bca8-3192318254d7\") " Jul 16 17:18:02.517670 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:02.517624 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59434dc4-0f1f-4425-bca8-3192318254d7-workload-socket\") pod \"59434dc4-0f1f-4425-bca8-3192318254d7\" (UID: \"59434dc4-0f1f-4425-bca8-3192318254d7\") " Jul 16 17:18:02.517670 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:02.517654 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59434dc4-0f1f-4425-bca8-3192318254d7-istio-token\") pod \"59434dc4-0f1f-4425-bca8-3192318254d7\" (UID: \"59434dc4-0f1f-4425-bca8-3192318254d7\") " Jul 16 17:18:02.518019 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:02.517680 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8s8g5\" (UniqueName: \"kubernetes.io/projected/59434dc4-0f1f-4425-bca8-3192318254d7-kube-api-access-8s8g5\") pod \"59434dc4-0f1f-4425-bca8-3192318254d7\" (UID: \"59434dc4-0f1f-4425-bca8-3192318254d7\") " Jul 16 17:18:02.518019 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:02.517780 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59434dc4-0f1f-4425-bca8-3192318254d7-istio-envoy\") pod \"59434dc4-0f1f-4425-bca8-3192318254d7\" (UID: \"59434dc4-0f1f-4425-bca8-3192318254d7\") " Jul 16 17:18:02.518019 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:02.517809 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59434dc4-0f1f-4425-bca8-3192318254d7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "59434dc4-0f1f-4425-bca8-3192318254d7" (UID: "59434dc4-0f1f-4425-bca8-3192318254d7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:18:02.518019 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:02.517968 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59434dc4-0f1f-4425-bca8-3192318254d7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "59434dc4-0f1f-4425-bca8-3192318254d7" (UID: "59434dc4-0f1f-4425-bca8-3192318254d7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:18:02.518208 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:02.518037 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59434dc4-0f1f-4425-bca8-3192318254d7-credential-socket\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:18:02.518208 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:02.518054 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59434dc4-0f1f-4425-bca8-3192318254d7-workload-socket\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:18:02.518208 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:02.518117 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59434dc4-0f1f-4425-bca8-3192318254d7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "59434dc4-0f1f-4425-bca8-3192318254d7" (UID: "59434dc4-0f1f-4425-bca8-3192318254d7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:18:02.518208 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:02.518114 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59434dc4-0f1f-4425-bca8-3192318254d7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "59434dc4-0f1f-4425-bca8-3192318254d7" (UID: "59434dc4-0f1f-4425-bca8-3192318254d7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:18:02.518413 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:02.518388 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59434dc4-0f1f-4425-bca8-3192318254d7-istio-data" (OuterVolumeSpecName: "istio-data") pod "59434dc4-0f1f-4425-bca8-3192318254d7" (UID: "59434dc4-0f1f-4425-bca8-3192318254d7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:18:02.519841 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:02.519812 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/59434dc4-0f1f-4425-bca8-3192318254d7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "59434dc4-0f1f-4425-bca8-3192318254d7" (UID: "59434dc4-0f1f-4425-bca8-3192318254d7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 17:18:02.519952 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:02.519834 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59434dc4-0f1f-4425-bca8-3192318254d7-istio-token" (OuterVolumeSpecName: "istio-token") pod "59434dc4-0f1f-4425-bca8-3192318254d7" (UID: "59434dc4-0f1f-4425-bca8-3192318254d7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:18:02.519952 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:02.519854 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59434dc4-0f1f-4425-bca8-3192318254d7-kube-api-access-8s8g5" (OuterVolumeSpecName: "kube-api-access-8s8g5") pod "59434dc4-0f1f-4425-bca8-3192318254d7" (UID: "59434dc4-0f1f-4425-bca8-3192318254d7"). InnerVolumeSpecName "kube-api-access-8s8g5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:18:02.519952 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:02.519940 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59434dc4-0f1f-4425-bca8-3192318254d7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "59434dc4-0f1f-4425-bca8-3192318254d7" (UID: "59434dc4-0f1f-4425-bca8-3192318254d7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:18:02.619350 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:02.619319 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59434dc4-0f1f-4425-bca8-3192318254d7-istiod-ca-cert\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:18:02.619350 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:02.619347 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59434dc4-0f1f-4425-bca8-3192318254d7-istio-token\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:18:02.619350 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:02.619356 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8s8g5\" (UniqueName: \"kubernetes.io/projected/59434dc4-0f1f-4425-bca8-3192318254d7-kube-api-access-8s8g5\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:18:02.619562 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:02.619366 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59434dc4-0f1f-4425-bca8-3192318254d7-istio-envoy\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:18:02.619562 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:02.619374 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59434dc4-0f1f-4425-bca8-3192318254d7-istio-data\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:18:02.619562 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:02.619381 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59434dc4-0f1f-4425-bca8-3192318254d7-istio-podinfo\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:18:02.619562 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:02.619389 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59434dc4-0f1f-4425-bca8-3192318254d7-workload-certs\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:18:03.046244 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:03.046151 2571 generic.go:356] "Generic (PLEG): container finished" podID="59434dc4-0f1f-4425-bca8-3192318254d7" containerID="af5c5d2c6076b457f663eb267b8fcc5f3a8766f7820e26279d86e14077d1522a" exitCode=0 Jul 16 17:18:03.046244 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:03.046202 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" event={"ID":"59434dc4-0f1f-4425-bca8-3192318254d7","Type":"ContainerDied","Data":"af5c5d2c6076b457f663eb267b8fcc5f3a8766f7820e26279d86e14077d1522a"} Jul 16 17:18:03.046244 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:03.046222 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" Jul 16 17:18:03.046481 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:03.046284 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn" event={"ID":"59434dc4-0f1f-4425-bca8-3192318254d7","Type":"ContainerDied","Data":"cc758b627c77fd82eb68bc01be81cf0200c9e5c2b2bf59d64a9be6f41407229c"} Jul 16 17:18:03.046481 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:03.046307 2571 scope.go:117] "RemoveContainer" containerID="af5c5d2c6076b457f663eb267b8fcc5f3a8766f7820e26279d86e14077d1522a" Jul 16 17:18:03.055836 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:03.055802 2571 scope.go:117] "RemoveContainer" containerID="af5c5d2c6076b457f663eb267b8fcc5f3a8766f7820e26279d86e14077d1522a" Jul 16 17:18:03.056084 ip-10-0-128-50 kubenswrapper[2571]: E0716 17:18:03.056063 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af5c5d2c6076b457f663eb267b8fcc5f3a8766f7820e26279d86e14077d1522a\": container with ID starting with af5c5d2c6076b457f663eb267b8fcc5f3a8766f7820e26279d86e14077d1522a not found: ID does not exist" containerID="af5c5d2c6076b457f663eb267b8fcc5f3a8766f7820e26279d86e14077d1522a" Jul 16 17:18:03.056147 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:03.056092 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af5c5d2c6076b457f663eb267b8fcc5f3a8766f7820e26279d86e14077d1522a"} err="failed to get container status \"af5c5d2c6076b457f663eb267b8fcc5f3a8766f7820e26279d86e14077d1522a\": rpc error: code = NotFound desc = could not find container \"af5c5d2c6076b457f663eb267b8fcc5f3a8766f7820e26279d86e14077d1522a\": container with ID starting with af5c5d2c6076b457f663eb267b8fcc5f3a8766f7820e26279d86e14077d1522a not found: ID does not exist" Jul 16 17:18:03.073986 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:03.073964 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn"] Jul 16 17:18:03.079755 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:03.079717 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-b593195d-gw-openshift-default-c84659b6c-dx9mn"] Jul 16 17:18:04.777278 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:04.777242 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59434dc4-0f1f-4425-bca8-3192318254d7" path="/var/lib/kubelet/pods/59434dc4-0f1f-4425-bca8-3192318254d7/volumes" Jul 16 17:18:10.734896 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:10.734860 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd"] Jul 16 17:18:10.735451 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:10.735401 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59434dc4-0f1f-4425-bca8-3192318254d7" containerName="istio-proxy" Jul 16 17:18:10.735451 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:10.735420 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="59434dc4-0f1f-4425-bca8-3192318254d7" containerName="istio-proxy" Jul 16 17:18:10.735565 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:10.735513 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="59434dc4-0f1f-4425-bca8-3192318254d7" containerName="istio-proxy" Jul 16 17:18:10.739811 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:10.739778 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" Jul 16 17:18:10.742820 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:10.742802 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-c527f463-openshift-default-dockercfg-pvdls\"" Jul 16 17:18:10.748447 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:10.748430 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 16 17:18:10.758994 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:10.758970 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd"] Jul 16 17:18:10.893793 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:10.893721 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed0e3345-032d-4e2c-a2a4-f7278051ceff-istiod-ca-cert\") pod \"e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd\" (UID: \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\") " pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" Jul 16 17:18:10.893983 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:10.893807 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed0e3345-032d-4e2c-a2a4-f7278051ceff-istio-envoy\") pod \"e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd\" (UID: \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\") " pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" Jul 16 17:18:10.893983 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:10.893830 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed0e3345-032d-4e2c-a2a4-f7278051ceff-credential-socket\") pod \"e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd\" (UID: \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\") " pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" Jul 16 17:18:10.893983 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:10.893883 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzj2s\" (UniqueName: \"kubernetes.io/projected/ed0e3345-032d-4e2c-a2a4-f7278051ceff-kube-api-access-tzj2s\") pod \"e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd\" (UID: \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\") " pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" Jul 16 17:18:10.893983 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:10.893902 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed0e3345-032d-4e2c-a2a4-f7278051ceff-istio-data\") pod \"e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd\" (UID: \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\") " pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" Jul 16 17:18:10.893983 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:10.893934 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed0e3345-032d-4e2c-a2a4-f7278051ceff-istio-podinfo\") pod \"e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd\" (UID: \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\") " pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" Jul 16 17:18:10.893983 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:10.893963 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed0e3345-032d-4e2c-a2a4-f7278051ceff-workload-socket\") pod \"e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd\" (UID: \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\") " pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" Jul 16 17:18:10.894286 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:10.893985 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed0e3345-032d-4e2c-a2a4-f7278051ceff-istio-token\") pod \"e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd\" (UID: \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\") " pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" Jul 16 17:18:10.894286 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:10.894056 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed0e3345-032d-4e2c-a2a4-f7278051ceff-workload-certs\") pod \"e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd\" (UID: \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\") " pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" Jul 16 17:18:10.994688 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:10.994582 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed0e3345-032d-4e2c-a2a4-f7278051ceff-workload-socket\") pod \"e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd\" (UID: \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\") " pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" Jul 16 17:18:10.994688 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:10.994637 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed0e3345-032d-4e2c-a2a4-f7278051ceff-istio-token\") pod \"e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd\" (UID: \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\") " pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" Jul 16 17:18:10.994927 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:10.994699 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed0e3345-032d-4e2c-a2a4-f7278051ceff-workload-certs\") pod \"e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd\" (UID: \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\") " pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" Jul 16 17:18:10.994927 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:10.994761 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed0e3345-032d-4e2c-a2a4-f7278051ceff-istiod-ca-cert\") pod \"e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd\" (UID: \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\") " pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" Jul 16 17:18:10.994927 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:10.994807 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed0e3345-032d-4e2c-a2a4-f7278051ceff-istio-envoy\") pod \"e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd\" (UID: \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\") " pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" Jul 16 17:18:10.994927 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:10.994833 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed0e3345-032d-4e2c-a2a4-f7278051ceff-credential-socket\") pod \"e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd\" (UID: \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\") " pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" Jul 16 17:18:10.994927 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:10.994859 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzj2s\" (UniqueName: \"kubernetes.io/projected/ed0e3345-032d-4e2c-a2a4-f7278051ceff-kube-api-access-tzj2s\") pod \"e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd\" (UID: \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\") " pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" Jul 16 17:18:10.994927 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:10.994888 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed0e3345-032d-4e2c-a2a4-f7278051ceff-istio-data\") pod \"e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd\" (UID: \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\") " pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" Jul 16 17:18:10.994927 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:10.994921 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed0e3345-032d-4e2c-a2a4-f7278051ceff-istio-podinfo\") pod \"e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd\" (UID: \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\") " pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" Jul 16 17:18:10.995427 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:10.995405 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed0e3345-032d-4e2c-a2a4-f7278051ceff-workload-certs\") pod \"e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd\" (UID: \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\") " pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" Jul 16 17:18:10.995507 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:10.995482 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed0e3345-032d-4e2c-a2a4-f7278051ceff-credential-socket\") pod \"e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd\" (UID: \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\") " pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" Jul 16 17:18:10.995800 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:10.995775 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed0e3345-032d-4e2c-a2a4-f7278051ceff-workload-socket\") pod \"e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd\" (UID: \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\") " pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" Jul 16 17:18:10.995886 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:10.995797 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed0e3345-032d-4e2c-a2a4-f7278051ceff-istiod-ca-cert\") pod \"e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd\" (UID: \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\") " pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" Jul 16 17:18:10.995886 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:10.995783 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed0e3345-032d-4e2c-a2a4-f7278051ceff-istio-data\") pod \"e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd\" (UID: \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\") " pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" Jul 16 17:18:10.998553 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:10.998528 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed0e3345-032d-4e2c-a2a4-f7278051ceff-istio-podinfo\") pod \"e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd\" (UID: \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\") " pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" Jul 16 17:18:10.998684 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:10.998645 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed0e3345-032d-4e2c-a2a4-f7278051ceff-istio-envoy\") pod \"e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd\" (UID: \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\") " pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" Jul 16 17:18:11.010314 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:11.010293 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed0e3345-032d-4e2c-a2a4-f7278051ceff-istio-token\") pod \"e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd\" (UID: \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\") " pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" Jul 16 17:18:11.010755 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:11.010715 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzj2s\" (UniqueName: \"kubernetes.io/projected/ed0e3345-032d-4e2c-a2a4-f7278051ceff-kube-api-access-tzj2s\") pod \"e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd\" (UID: \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\") " pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" Jul 16 17:18:11.050900 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:11.050865 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" Jul 16 17:18:11.192370 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:11.192340 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd"] Jul 16 17:18:11.194776 ip-10-0-128-50 kubenswrapper[2571]: W0716 17:18:11.194705 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded0e3345_032d_4e2c_a2a4_f7278051ceff.slice/crio-7515a774dd976dfffc663d76d8b40c2473cfd6cb8b607f39022caf4a1ab80e4c WatchSource:0}: Error finding container 7515a774dd976dfffc663d76d8b40c2473cfd6cb8b607f39022caf4a1ab80e4c: Status 404 returned error can't find the container with id 7515a774dd976dfffc663d76d8b40c2473cfd6cb8b607f39022caf4a1ab80e4c Jul 16 17:18:11.197570 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:11.197533 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 17:18:11.197667 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:11.197619 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 17:18:11.197745 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:11.197662 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 17:18:12.094965 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:12.094924 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" event={"ID":"ed0e3345-032d-4e2c-a2a4-f7278051ceff","Type":"ContainerStarted","Data":"cd9d60c3f89ac97883619e674992a49fb1921eea39e3d8bf940f11fdb70e6acf"} Jul 16 17:18:12.095414 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:12.094989 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" event={"ID":"ed0e3345-032d-4e2c-a2a4-f7278051ceff","Type":"ContainerStarted","Data":"7515a774dd976dfffc663d76d8b40c2473cfd6cb8b607f39022caf4a1ab80e4c"} Jul 16 17:18:12.127270 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:12.127219 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" podStartSLOduration=2.127205132 podStartE2EDuration="2.127205132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:18:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:18:12.124877221 +0000 UTC m=+1643.866452540" watchObservedRunningTime="2026-07-16 17:18:12.127205132 +0000 UTC m=+1643.868780426" Jul 16 17:18:13.051411 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:13.051379 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" Jul 16 17:18:13.056861 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:13.056836 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" Jul 16 17:18:13.100439 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:13.100401 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" Jul 16 17:18:13.101414 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:13.101394 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" Jul 16 17:18:16.464858 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:16.464821 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd"] Jul 16 17:18:16.465296 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:16.465005 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" podUID="ed0e3345-032d-4e2c-a2a4-f7278051ceff" containerName="istio-proxy" containerID="cri-o://cd9d60c3f89ac97883619e674992a49fb1921eea39e3d8bf940f11fdb70e6acf" gracePeriod=30 Jul 16 17:18:22.617661 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:22.617637 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" Jul 16 17:18:22.694962 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:22.694931 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed0e3345-032d-4e2c-a2a4-f7278051ceff-istio-podinfo\") pod \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\" (UID: \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\") " Jul 16 17:18:22.695103 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:22.694984 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed0e3345-032d-4e2c-a2a4-f7278051ceff-istiod-ca-cert\") pod \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\" (UID: \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\") " Jul 16 17:18:22.695103 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:22.695017 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed0e3345-032d-4e2c-a2a4-f7278051ceff-istio-token\") pod \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\" (UID: \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\") " Jul 16 17:18:22.695103 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:22.695042 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed0e3345-032d-4e2c-a2a4-f7278051ceff-workload-socket\") pod \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\" (UID: \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\") " Jul 16 17:18:22.695103 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:22.695088 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed0e3345-032d-4e2c-a2a4-f7278051ceff-istio-data\") pod \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\" (UID: \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\") " Jul 16 17:18:22.695315 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:22.695116 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed0e3345-032d-4e2c-a2a4-f7278051ceff-istio-envoy\") pod \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\" (UID: \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\") " Jul 16 17:18:22.695315 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:22.695142 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed0e3345-032d-4e2c-a2a4-f7278051ceff-workload-certs\") pod \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\" (UID: \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\") " Jul 16 17:18:22.695315 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:22.695183 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed0e3345-032d-4e2c-a2a4-f7278051ceff-credential-socket\") pod \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\" (UID: \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\") " Jul 16 17:18:22.695315 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:22.695219 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tzj2s\" (UniqueName: \"kubernetes.io/projected/ed0e3345-032d-4e2c-a2a4-f7278051ceff-kube-api-access-tzj2s\") pod \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\" (UID: \"ed0e3345-032d-4e2c-a2a4-f7278051ceff\") " Jul 16 17:18:22.695524 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:22.695354 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed0e3345-032d-4e2c-a2a4-f7278051ceff-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ed0e3345-032d-4e2c-a2a4-f7278051ceff" (UID: "ed0e3345-032d-4e2c-a2a4-f7278051ceff"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:18:22.695583 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:22.695525 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed0e3345-032d-4e2c-a2a4-f7278051ceff-workload-socket\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:18:22.695583 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:22.695567 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed0e3345-032d-4e2c-a2a4-f7278051ceff-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ed0e3345-032d-4e2c-a2a4-f7278051ceff" (UID: "ed0e3345-032d-4e2c-a2a4-f7278051ceff"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:18:22.695761 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:22.695720 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed0e3345-032d-4e2c-a2a4-f7278051ceff-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ed0e3345-032d-4e2c-a2a4-f7278051ceff" (UID: "ed0e3345-032d-4e2c-a2a4-f7278051ceff"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:18:22.695831 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:22.695746 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed0e3345-032d-4e2c-a2a4-f7278051ceff-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ed0e3345-032d-4e2c-a2a4-f7278051ceff" (UID: "ed0e3345-032d-4e2c-a2a4-f7278051ceff"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:18:22.696571 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:22.696536 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed0e3345-032d-4e2c-a2a4-f7278051ceff-istio-data" (OuterVolumeSpecName: "istio-data") pod "ed0e3345-032d-4e2c-a2a4-f7278051ceff" (UID: "ed0e3345-032d-4e2c-a2a4-f7278051ceff"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:18:22.697200 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:22.697178 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ed0e3345-032d-4e2c-a2a4-f7278051ceff-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ed0e3345-032d-4e2c-a2a4-f7278051ceff" (UID: "ed0e3345-032d-4e2c-a2a4-f7278051ceff"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 17:18:22.697272 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:22.697196 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed0e3345-032d-4e2c-a2a4-f7278051ceff-istio-token" (OuterVolumeSpecName: "istio-token") pod "ed0e3345-032d-4e2c-a2a4-f7278051ceff" (UID: "ed0e3345-032d-4e2c-a2a4-f7278051ceff"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:18:22.697633 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:22.697613 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed0e3345-032d-4e2c-a2a4-f7278051ceff-kube-api-access-tzj2s" (OuterVolumeSpecName: "kube-api-access-tzj2s") pod "ed0e3345-032d-4e2c-a2a4-f7278051ceff" (UID: "ed0e3345-032d-4e2c-a2a4-f7278051ceff"). InnerVolumeSpecName "kube-api-access-tzj2s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:18:22.697747 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:22.697683 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed0e3345-032d-4e2c-a2a4-f7278051ceff-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ed0e3345-032d-4e2c-a2a4-f7278051ceff" (UID: "ed0e3345-032d-4e2c-a2a4-f7278051ceff"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:18:22.796261 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:22.796237 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed0e3345-032d-4e2c-a2a4-f7278051ceff-istio-podinfo\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:18:22.796361 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:22.796267 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed0e3345-032d-4e2c-a2a4-f7278051ceff-istiod-ca-cert\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:18:22.796361 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:22.796281 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed0e3345-032d-4e2c-a2a4-f7278051ceff-istio-token\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:18:22.796361 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:22.796293 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed0e3345-032d-4e2c-a2a4-f7278051ceff-istio-data\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:18:22.796361 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:22.796305 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed0e3345-032d-4e2c-a2a4-f7278051ceff-istio-envoy\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:18:22.796361 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:22.796316 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed0e3345-032d-4e2c-a2a4-f7278051ceff-workload-certs\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:18:22.796361 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:22.796328 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed0e3345-032d-4e2c-a2a4-f7278051ceff-credential-socket\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:18:22.796361 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:22.796341 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzj2s\" (UniqueName: \"kubernetes.io/projected/ed0e3345-032d-4e2c-a2a4-f7278051ceff-kube-api-access-tzj2s\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:18:23.158835 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:23.158749 2571 generic.go:356] "Generic (PLEG): container finished" podID="ed0e3345-032d-4e2c-a2a4-f7278051ceff" containerID="cd9d60c3f89ac97883619e674992a49fb1921eea39e3d8bf940f11fdb70e6acf" exitCode=0 Jul 16 17:18:23.158835 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:23.158794 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" Jul 16 17:18:23.159008 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:23.158832 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" event={"ID":"ed0e3345-032d-4e2c-a2a4-f7278051ceff","Type":"ContainerDied","Data":"cd9d60c3f89ac97883619e674992a49fb1921eea39e3d8bf940f11fdb70e6acf"} Jul 16 17:18:23.159008 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:23.158885 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd" event={"ID":"ed0e3345-032d-4e2c-a2a4-f7278051ceff","Type":"ContainerDied","Data":"7515a774dd976dfffc663d76d8b40c2473cfd6cb8b607f39022caf4a1ab80e4c"} Jul 16 17:18:23.159008 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:23.158910 2571 scope.go:117] "RemoveContainer" containerID="cd9d60c3f89ac97883619e674992a49fb1921eea39e3d8bf940f11fdb70e6acf" Jul 16 17:18:23.167620 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:23.167603 2571 scope.go:117] "RemoveContainer" containerID="cd9d60c3f89ac97883619e674992a49fb1921eea39e3d8bf940f11fdb70e6acf" Jul 16 17:18:23.167907 ip-10-0-128-50 kubenswrapper[2571]: E0716 17:18:23.167891 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd9d60c3f89ac97883619e674992a49fb1921eea39e3d8bf940f11fdb70e6acf\": container with ID starting with cd9d60c3f89ac97883619e674992a49fb1921eea39e3d8bf940f11fdb70e6acf not found: ID does not exist" containerID="cd9d60c3f89ac97883619e674992a49fb1921eea39e3d8bf940f11fdb70e6acf" Jul 16 17:18:23.167957 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:23.167914 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd9d60c3f89ac97883619e674992a49fb1921eea39e3d8bf940f11fdb70e6acf"} err="failed to get container status \"cd9d60c3f89ac97883619e674992a49fb1921eea39e3d8bf940f11fdb70e6acf\": rpc error: code = NotFound desc = could not find container \"cd9d60c3f89ac97883619e674992a49fb1921eea39e3d8bf940f11fdb70e6acf\": container with ID starting with cd9d60c3f89ac97883619e674992a49fb1921eea39e3d8bf940f11fdb70e6acf not found: ID does not exist" Jul 16 17:18:23.182173 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:23.182151 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd"] Jul 16 17:18:23.189108 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:23.189088 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-c527f463-openshift-default-849c4b6f6c-8d9hd"] Jul 16 17:18:24.776973 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:18:24.776945 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed0e3345-032d-4e2c-a2a4-f7278051ceff" path="/var/lib/kubelet/pods/ed0e3345-032d-4e2c-a2a4-f7278051ceff/volumes" Jul 16 17:20:20.951823 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:20.951793 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq"] Jul 16 17:20:20.952227 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:20.952114 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed0e3345-032d-4e2c-a2a4-f7278051ceff" containerName="istio-proxy" Jul 16 17:20:20.952227 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:20.952124 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed0e3345-032d-4e2c-a2a4-f7278051ceff" containerName="istio-proxy" Jul 16 17:20:20.952227 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:20.952194 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed0e3345-032d-4e2c-a2a4-f7278051ceff" containerName="istio-proxy" Jul 16 17:20:20.955225 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:20.955205 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:20.969031 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:20.969014 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 16 17:20:20.969189 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:20.969173 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-0c09a637-openshift-default-dockercfg-rds8p\"" Jul 16 17:20:21.038032 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:21.038008 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq"] Jul 16 17:20:21.136342 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:21.136320 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5410b4e8-f4d2-439e-a5a8-970a04a4780e-istiod-ca-cert\") pod \"e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq\" (UID: \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\") " pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:21.136476 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:21.136348 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5410b4e8-f4d2-439e-a5a8-970a04a4780e-workload-socket\") pod \"e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq\" (UID: \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\") " pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:21.136476 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:21.136372 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5410b4e8-f4d2-439e-a5a8-970a04a4780e-istio-token\") pod \"e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq\" (UID: \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\") " pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:21.136476 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:21.136388 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5410b4e8-f4d2-439e-a5a8-970a04a4780e-workload-certs\") pod \"e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq\" (UID: \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\") " pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:21.136645 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:21.136478 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5410b4e8-f4d2-439e-a5a8-970a04a4780e-istio-envoy\") pod \"e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq\" (UID: \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\") " pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:21.136645 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:21.136508 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5410b4e8-f4d2-439e-a5a8-970a04a4780e-istio-data\") pod \"e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq\" (UID: \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\") " pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:21.136645 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:21.136538 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5410b4e8-f4d2-439e-a5a8-970a04a4780e-istio-podinfo\") pod \"e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq\" (UID: \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\") " pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:21.136645 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:21.136576 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cgww\" (UniqueName: \"kubernetes.io/projected/5410b4e8-f4d2-439e-a5a8-970a04a4780e-kube-api-access-7cgww\") pod \"e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq\" (UID: \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\") " pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:21.136645 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:21.136603 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5410b4e8-f4d2-439e-a5a8-970a04a4780e-credential-socket\") pod \"e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq\" (UID: \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\") " pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:21.237895 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:21.237830 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5410b4e8-f4d2-439e-a5a8-970a04a4780e-workload-certs\") pod \"e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq\" (UID: \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\") " pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:21.237895 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:21.237870 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5410b4e8-f4d2-439e-a5a8-970a04a4780e-istio-envoy\") pod \"e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq\" (UID: \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\") " pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:21.237895 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:21.237888 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5410b4e8-f4d2-439e-a5a8-970a04a4780e-istio-data\") pod \"e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq\" (UID: \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\") " pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:21.238102 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:21.237911 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5410b4e8-f4d2-439e-a5a8-970a04a4780e-istio-podinfo\") pod \"e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq\" (UID: \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\") " pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:21.238102 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:21.237939 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cgww\" (UniqueName: \"kubernetes.io/projected/5410b4e8-f4d2-439e-a5a8-970a04a4780e-kube-api-access-7cgww\") pod \"e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq\" (UID: \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\") " pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:21.238102 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:21.237963 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5410b4e8-f4d2-439e-a5a8-970a04a4780e-credential-socket\") pod \"e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq\" (UID: \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\") " pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:21.238102 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:21.238036 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5410b4e8-f4d2-439e-a5a8-970a04a4780e-istiod-ca-cert\") pod \"e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq\" (UID: \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\") " pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:21.238102 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:21.238075 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5410b4e8-f4d2-439e-a5a8-970a04a4780e-workload-socket\") pod \"e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq\" (UID: \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\") " pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:21.238359 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:21.238107 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5410b4e8-f4d2-439e-a5a8-970a04a4780e-istio-token\") pod \"e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq\" (UID: \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\") " pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:21.238359 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:21.238242 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5410b4e8-f4d2-439e-a5a8-970a04a4780e-workload-certs\") pod \"e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq\" (UID: \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\") " pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:21.238359 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:21.238289 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5410b4e8-f4d2-439e-a5a8-970a04a4780e-istio-data\") pod \"e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq\" (UID: \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\") " pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:21.238519 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:21.238425 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5410b4e8-f4d2-439e-a5a8-970a04a4780e-credential-socket\") pod \"e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq\" (UID: \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\") " pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:21.238934 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:21.238910 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5410b4e8-f4d2-439e-a5a8-970a04a4780e-workload-socket\") pod \"e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq\" (UID: \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\") " pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:21.239055 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:21.239030 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5410b4e8-f4d2-439e-a5a8-970a04a4780e-istiod-ca-cert\") pod \"e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq\" (UID: \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\") " pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:21.240241 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:21.240222 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5410b4e8-f4d2-439e-a5a8-970a04a4780e-istio-envoy\") pod \"e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq\" (UID: \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\") " pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:21.240371 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:21.240354 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5410b4e8-f4d2-439e-a5a8-970a04a4780e-istio-podinfo\") pod \"e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq\" (UID: \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\") " pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:21.268111 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:21.268086 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5410b4e8-f4d2-439e-a5a8-970a04a4780e-istio-token\") pod \"e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq\" (UID: \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\") " pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:21.268794 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:21.268780 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cgww\" (UniqueName: \"kubernetes.io/projected/5410b4e8-f4d2-439e-a5a8-970a04a4780e-kube-api-access-7cgww\") pod \"e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq\" (UID: \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\") " pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:21.564609 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:21.564547 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:21.704286 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:21.704254 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq"] Jul 16 17:20:21.707511 ip-10-0-128-50 kubenswrapper[2571]: W0716 17:20:21.707482 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5410b4e8_f4d2_439e_a5a8_970a04a4780e.slice/crio-9eff41884981b47b5095d8db3aed12f77ea675a6baeabd20fe9e06ce747e43b7 WatchSource:0}: Error finding container 9eff41884981b47b5095d8db3aed12f77ea675a6baeabd20fe9e06ce747e43b7: Status 404 returned error can't find the container with id 9eff41884981b47b5095d8db3aed12f77ea675a6baeabd20fe9e06ce747e43b7 Jul 16 17:20:21.709602 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:21.709569 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 17:20:21.709669 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:21.709640 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 17:20:21.709712 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:21.709674 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 17:20:21.769785 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:21.769756 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" event={"ID":"5410b4e8-f4d2-439e-a5a8-970a04a4780e","Type":"ContainerStarted","Data":"9eff41884981b47b5095d8db3aed12f77ea675a6baeabd20fe9e06ce747e43b7"} Jul 16 17:20:22.776584 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:22.776549 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" event={"ID":"5410b4e8-f4d2-439e-a5a8-970a04a4780e","Type":"ContainerStarted","Data":"8f9ffb3d6b4c3b2b4d1c9faee21518e7e3ed9d14aea1f320a34fa07945f4657b"} Jul 16 17:20:22.808715 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:22.808670 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" podStartSLOduration=2.80865472 podStartE2EDuration="2.80865472s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:20:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:20:22.807538951 +0000 UTC m=+1774.549114246" watchObservedRunningTime="2026-07-16 17:20:22.80865472 +0000 UTC m=+1774.550230015" Jul 16 17:20:23.565451 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:23.565414 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:23.570210 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:23.570186 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:23.781346 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:23.781309 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:23.782073 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:23.782055 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:37.862327 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:37.862295 2571 generic.go:356] "Generic (PLEG): container finished" podID="5410b4e8-f4d2-439e-a5a8-970a04a4780e" containerID="8f9ffb3d6b4c3b2b4d1c9faee21518e7e3ed9d14aea1f320a34fa07945f4657b" exitCode=0 Jul 16 17:20:37.862837 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:37.862383 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" event={"ID":"5410b4e8-f4d2-439e-a5a8-970a04a4780e","Type":"ContainerDied","Data":"8f9ffb3d6b4c3b2b4d1c9faee21518e7e3ed9d14aea1f320a34fa07945f4657b"} Jul 16 17:20:37.862920 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:37.862848 2571 scope.go:117] "RemoveContainer" containerID="8f9ffb3d6b4c3b2b4d1c9faee21518e7e3ed9d14aea1f320a34fa07945f4657b" Jul 16 17:20:37.863845 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:37.863817 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 17:20:37.863918 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:37.863899 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 17:20:37.864002 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:37.863984 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 17:20:38.781514 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:38.781483 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:38.869276 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:38.869247 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" event={"ID":"5410b4e8-f4d2-439e-a5a8-970a04a4780e","Type":"ContainerStarted","Data":"977529efa447335c01ec40e94614ff85dc664c64278a2d386e1f8767ad3bdf3b"} Jul 16 17:20:39.565437 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:39.565405 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:39.566428 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:39.566404 2571 patch_prober.go:28] interesting pod/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 16 17:20:39.566559 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:39.566454 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" podUID="5410b4e8-f4d2-439e-a5a8-970a04a4780e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 16 17:20:40.565068 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:40.565036 2571 patch_prober.go:28] interesting pod/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 16 17:20:40.565448 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:40.565094 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" podUID="5410b4e8-f4d2-439e-a5a8-970a04a4780e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 16 17:20:41.564928 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:41.564839 2571 patch_prober.go:28] interesting pod/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 16 17:20:41.564928 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:41.564895 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" podUID="5410b4e8-f4d2-439e-a5a8-970a04a4780e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 16 17:20:42.569583 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:42.569553 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:42.570036 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:42.569899 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:42.570485 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:42.570465 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:48.869287 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:48.869258 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r8cn4_f637d961-672d-4aa2-a49a-fd021c2d4df2/console-operator/2.log" Jul 16 17:20:48.872700 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:48.872676 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r8cn4_f637d961-672d-4aa2-a49a-fd021c2d4df2/console-operator/2.log" Jul 16 17:20:48.877723 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:48.877701 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qd7bg_cfbe9f0b-b759-4064-9332-3b48c547c003/ovn-acl-logging/0.log" Jul 16 17:20:48.880870 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:48.880852 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qd7bg_cfbe9f0b-b759-4064-9332-3b48c547c003/ovn-acl-logging/0.log" Jul 16 17:20:54.350801 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:54.350770 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq"] Jul 16 17:20:54.351210 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:54.350958 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" podUID="5410b4e8-f4d2-439e-a5a8-970a04a4780e" containerName="istio-proxy" containerID="cri-o://977529efa447335c01ec40e94614ff85dc664c64278a2d386e1f8767ad3bdf3b" gracePeriod=30 Jul 16 17:20:57.572236 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:57.572205 2571 patch_prober.go:28] interesting pod/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused Jul 16 17:20:57.572569 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:57.572267 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" podUID="5410b4e8-f4d2-439e-a5a8-970a04a4780e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 17:20:59.594527 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:59.594499 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:59.740325 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:59.740303 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5410b4e8-f4d2-439e-a5a8-970a04a4780e-workload-certs\") pod \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\" (UID: \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\") " Jul 16 17:20:59.740477 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:59.740338 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5410b4e8-f4d2-439e-a5a8-970a04a4780e-istio-token\") pod \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\" (UID: \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\") " Jul 16 17:20:59.740477 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:59.740356 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5410b4e8-f4d2-439e-a5a8-970a04a4780e-istio-envoy\") pod \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\" (UID: \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\") " Jul 16 17:20:59.740477 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:59.740372 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5410b4e8-f4d2-439e-a5a8-970a04a4780e-istio-data\") pod \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\" (UID: \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\") " Jul 16 17:20:59.740477 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:59.740450 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5410b4e8-f4d2-439e-a5a8-970a04a4780e-istio-podinfo\") pod \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\" (UID: \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\") " Jul 16 17:20:59.740697 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:59.740503 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5410b4e8-f4d2-439e-a5a8-970a04a4780e-workload-socket\") pod \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\" (UID: \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\") " Jul 16 17:20:59.740697 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:59.740528 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7cgww\" (UniqueName: \"kubernetes.io/projected/5410b4e8-f4d2-439e-a5a8-970a04a4780e-kube-api-access-7cgww\") pod \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\" (UID: \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\") " Jul 16 17:20:59.740697 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:59.740576 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5410b4e8-f4d2-439e-a5a8-970a04a4780e-istiod-ca-cert\") pod \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\" (UID: \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\") " Jul 16 17:20:59.740697 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:59.740620 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5410b4e8-f4d2-439e-a5a8-970a04a4780e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5410b4e8-f4d2-439e-a5a8-970a04a4780e" (UID: "5410b4e8-f4d2-439e-a5a8-970a04a4780e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:20:59.740697 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:59.740628 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5410b4e8-f4d2-439e-a5a8-970a04a4780e-credential-socket\") pod \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\" (UID: \"5410b4e8-f4d2-439e-a5a8-970a04a4780e\") " Jul 16 17:20:59.740992 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:59.740767 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5410b4e8-f4d2-439e-a5a8-970a04a4780e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5410b4e8-f4d2-439e-a5a8-970a04a4780e" (UID: "5410b4e8-f4d2-439e-a5a8-970a04a4780e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:20:59.740992 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:59.740942 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5410b4e8-f4d2-439e-a5a8-970a04a4780e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5410b4e8-f4d2-439e-a5a8-970a04a4780e" (UID: "5410b4e8-f4d2-439e-a5a8-970a04a4780e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:20:59.741102 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:59.741018 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5410b4e8-f4d2-439e-a5a8-970a04a4780e-workload-socket\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:20:59.741102 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:59.741039 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5410b4e8-f4d2-439e-a5a8-970a04a4780e-workload-certs\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:20:59.741102 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:59.741091 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5410b4e8-f4d2-439e-a5a8-970a04a4780e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5410b4e8-f4d2-439e-a5a8-970a04a4780e" (UID: "5410b4e8-f4d2-439e-a5a8-970a04a4780e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:20:59.741305 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:59.741272 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5410b4e8-f4d2-439e-a5a8-970a04a4780e-istio-data" (OuterVolumeSpecName: "istio-data") pod "5410b4e8-f4d2-439e-a5a8-970a04a4780e" (UID: "5410b4e8-f4d2-439e-a5a8-970a04a4780e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:20:59.742737 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:59.742701 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5410b4e8-f4d2-439e-a5a8-970a04a4780e-kube-api-access-7cgww" (OuterVolumeSpecName: "kube-api-access-7cgww") pod "5410b4e8-f4d2-439e-a5a8-970a04a4780e" (UID: "5410b4e8-f4d2-439e-a5a8-970a04a4780e"). InnerVolumeSpecName "kube-api-access-7cgww". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:20:59.742883 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:59.742863 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5410b4e8-f4d2-439e-a5a8-970a04a4780e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5410b4e8-f4d2-439e-a5a8-970a04a4780e" (UID: "5410b4e8-f4d2-439e-a5a8-970a04a4780e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:20:59.742953 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:59.742867 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5410b4e8-f4d2-439e-a5a8-970a04a4780e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5410b4e8-f4d2-439e-a5a8-970a04a4780e" (UID: "5410b4e8-f4d2-439e-a5a8-970a04a4780e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 17:20:59.743014 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:59.742982 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5410b4e8-f4d2-439e-a5a8-970a04a4780e-istio-token" (OuterVolumeSpecName: "istio-token") pod "5410b4e8-f4d2-439e-a5a8-970a04a4780e" (UID: "5410b4e8-f4d2-439e-a5a8-970a04a4780e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:20:59.841547 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:59.841525 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5410b4e8-f4d2-439e-a5a8-970a04a4780e-istiod-ca-cert\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:20:59.841547 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:59.841544 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5410b4e8-f4d2-439e-a5a8-970a04a4780e-credential-socket\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:20:59.841669 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:59.841554 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5410b4e8-f4d2-439e-a5a8-970a04a4780e-istio-token\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:20:59.841669 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:59.841562 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5410b4e8-f4d2-439e-a5a8-970a04a4780e-istio-envoy\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:20:59.841669 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:59.841570 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5410b4e8-f4d2-439e-a5a8-970a04a4780e-istio-data\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:20:59.841669 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:59.841578 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5410b4e8-f4d2-439e-a5a8-970a04a4780e-istio-podinfo\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:20:59.841669 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:59.841585 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7cgww\" (UniqueName: \"kubernetes.io/projected/5410b4e8-f4d2-439e-a5a8-970a04a4780e-kube-api-access-7cgww\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:20:59.976889 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:59.976862 2571 generic.go:356] "Generic (PLEG): container finished" podID="5410b4e8-f4d2-439e-a5a8-970a04a4780e" containerID="977529efa447335c01ec40e94614ff85dc664c64278a2d386e1f8767ad3bdf3b" exitCode=0 Jul 16 17:20:59.976973 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:59.976909 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" Jul 16 17:20:59.976973 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:59.976946 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" event={"ID":"5410b4e8-f4d2-439e-a5a8-970a04a4780e","Type":"ContainerDied","Data":"977529efa447335c01ec40e94614ff85dc664c64278a2d386e1f8767ad3bdf3b"} Jul 16 17:20:59.977045 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:59.976995 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq" event={"ID":"5410b4e8-f4d2-439e-a5a8-970a04a4780e","Type":"ContainerDied","Data":"9eff41884981b47b5095d8db3aed12f77ea675a6baeabd20fe9e06ce747e43b7"} Jul 16 17:20:59.977045 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:59.977017 2571 scope.go:117] "RemoveContainer" containerID="977529efa447335c01ec40e94614ff85dc664c64278a2d386e1f8767ad3bdf3b" Jul 16 17:20:59.985710 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:59.985687 2571 scope.go:117] "RemoveContainer" containerID="8f9ffb3d6b4c3b2b4d1c9faee21518e7e3ed9d14aea1f320a34fa07945f4657b" Jul 16 17:20:59.993974 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:59.993943 2571 scope.go:117] "RemoveContainer" containerID="977529efa447335c01ec40e94614ff85dc664c64278a2d386e1f8767ad3bdf3b" Jul 16 17:20:59.998576 ip-10-0-128-50 kubenswrapper[2571]: E0716 17:20:59.994411 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"977529efa447335c01ec40e94614ff85dc664c64278a2d386e1f8767ad3bdf3b\": container with ID starting with 977529efa447335c01ec40e94614ff85dc664c64278a2d386e1f8767ad3bdf3b not found: ID does not exist" containerID="977529efa447335c01ec40e94614ff85dc664c64278a2d386e1f8767ad3bdf3b" Jul 16 17:20:59.998576 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:59.994458 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"977529efa447335c01ec40e94614ff85dc664c64278a2d386e1f8767ad3bdf3b"} err="failed to get container status \"977529efa447335c01ec40e94614ff85dc664c64278a2d386e1f8767ad3bdf3b\": rpc error: code = NotFound desc = could not find container \"977529efa447335c01ec40e94614ff85dc664c64278a2d386e1f8767ad3bdf3b\": container with ID starting with 977529efa447335c01ec40e94614ff85dc664c64278a2d386e1f8767ad3bdf3b not found: ID does not exist" Jul 16 17:20:59.998576 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:59.994487 2571 scope.go:117] "RemoveContainer" containerID="8f9ffb3d6b4c3b2b4d1c9faee21518e7e3ed9d14aea1f320a34fa07945f4657b" Jul 16 17:20:59.998810 ip-10-0-128-50 kubenswrapper[2571]: E0716 17:20:59.998784 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f9ffb3d6b4c3b2b4d1c9faee21518e7e3ed9d14aea1f320a34fa07945f4657b\": container with ID starting with 8f9ffb3d6b4c3b2b4d1c9faee21518e7e3ed9d14aea1f320a34fa07945f4657b not found: ID does not exist" containerID="8f9ffb3d6b4c3b2b4d1c9faee21518e7e3ed9d14aea1f320a34fa07945f4657b" Jul 16 17:20:59.998860 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:20:59.998821 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f9ffb3d6b4c3b2b4d1c9faee21518e7e3ed9d14aea1f320a34fa07945f4657b"} err="failed to get container status \"8f9ffb3d6b4c3b2b4d1c9faee21518e7e3ed9d14aea1f320a34fa07945f4657b\": rpc error: code = NotFound desc = could not find container \"8f9ffb3d6b4c3b2b4d1c9faee21518e7e3ed9d14aea1f320a34fa07945f4657b\": container with ID starting with 8f9ffb3d6b4c3b2b4d1c9faee21518e7e3ed9d14aea1f320a34fa07945f4657b not found: ID does not exist" Jul 16 17:21:00.007677 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:21:00.007655 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq"] Jul 16 17:21:00.012828 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:21:00.012808 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-0c09a637-openshift-default-6c5d958bbc-clcdq"] Jul 16 17:21:00.777471 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:21:00.777441 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5410b4e8-f4d2-439e-a5a8-970a04a4780e" path="/var/lib/kubelet/pods/5410b4e8-f4d2-439e-a5a8-970a04a4780e/volumes" Jul 16 17:22:17.547163 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:17.547086 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54"] Jul 16 17:22:17.547951 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:17.547550 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5410b4e8-f4d2-439e-a5a8-970a04a4780e" containerName="istio-proxy" Jul 16 17:22:17.547951 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:17.547568 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5410b4e8-f4d2-439e-a5a8-970a04a4780e" containerName="istio-proxy" Jul 16 17:22:17.547951 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:17.547584 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5410b4e8-f4d2-439e-a5a8-970a04a4780e" containerName="istio-proxy" Jul 16 17:22:17.547951 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:17.547593 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5410b4e8-f4d2-439e-a5a8-970a04a4780e" containerName="istio-proxy" Jul 16 17:22:17.547951 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:17.547700 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="5410b4e8-f4d2-439e-a5a8-970a04a4780e" containerName="istio-proxy" Jul 16 17:22:17.550879 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:17.550859 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" Jul 16 17:22:17.555579 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:17.555557 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 16 17:22:17.555700 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:17.555622 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-7b9f8f-openshift-default-dockercfg-7pg4t\"" Jul 16 17:22:17.581804 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:17.581782 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54"] Jul 16 17:22:17.610465 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:17.610442 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/347bfee3-5a7a-48b1-a227-0689c2790b5c-istiod-ca-cert\") pod \"e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54\" (UID: \"347bfee3-5a7a-48b1-a227-0689c2790b5c\") " pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" Jul 16 17:22:17.610591 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:17.610479 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/347bfee3-5a7a-48b1-a227-0689c2790b5c-istio-envoy\") pod \"e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54\" (UID: \"347bfee3-5a7a-48b1-a227-0689c2790b5c\") " pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" Jul 16 17:22:17.610591 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:17.610498 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/347bfee3-5a7a-48b1-a227-0689c2790b5c-workload-certs\") pod \"e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54\" (UID: \"347bfee3-5a7a-48b1-a227-0689c2790b5c\") " pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" Jul 16 17:22:17.610591 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:17.610514 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpd5l\" (UniqueName: \"kubernetes.io/projected/347bfee3-5a7a-48b1-a227-0689c2790b5c-kube-api-access-tpd5l\") pod \"e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54\" (UID: \"347bfee3-5a7a-48b1-a227-0689c2790b5c\") " pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" Jul 16 17:22:17.610591 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:17.610536 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/347bfee3-5a7a-48b1-a227-0689c2790b5c-workload-socket\") pod \"e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54\" (UID: \"347bfee3-5a7a-48b1-a227-0689c2790b5c\") " pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" Jul 16 17:22:17.610785 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:17.610586 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/347bfee3-5a7a-48b1-a227-0689c2790b5c-credential-socket\") pod \"e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54\" (UID: \"347bfee3-5a7a-48b1-a227-0689c2790b5c\") " pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" Jul 16 17:22:17.610785 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:17.610635 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/347bfee3-5a7a-48b1-a227-0689c2790b5c-istio-data\") pod \"e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54\" (UID: \"347bfee3-5a7a-48b1-a227-0689c2790b5c\") " pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" Jul 16 17:22:17.610785 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:17.610672 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/347bfee3-5a7a-48b1-a227-0689c2790b5c-istio-podinfo\") pod \"e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54\" (UID: \"347bfee3-5a7a-48b1-a227-0689c2790b5c\") " pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" Jul 16 17:22:17.610785 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:17.610698 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/347bfee3-5a7a-48b1-a227-0689c2790b5c-istio-token\") pod \"e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54\" (UID: \"347bfee3-5a7a-48b1-a227-0689c2790b5c\") " pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" Jul 16 17:22:17.711587 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:17.711560 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/347bfee3-5a7a-48b1-a227-0689c2790b5c-istio-envoy\") pod \"e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54\" (UID: \"347bfee3-5a7a-48b1-a227-0689c2790b5c\") " pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" Jul 16 17:22:17.711692 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:17.711601 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/347bfee3-5a7a-48b1-a227-0689c2790b5c-workload-certs\") pod \"e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54\" (UID: \"347bfee3-5a7a-48b1-a227-0689c2790b5c\") " pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" Jul 16 17:22:17.711692 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:17.711617 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpd5l\" (UniqueName: \"kubernetes.io/projected/347bfee3-5a7a-48b1-a227-0689c2790b5c-kube-api-access-tpd5l\") pod \"e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54\" (UID: \"347bfee3-5a7a-48b1-a227-0689c2790b5c\") " pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" Jul 16 17:22:17.711692 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:17.711635 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/347bfee3-5a7a-48b1-a227-0689c2790b5c-workload-socket\") pod \"e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54\" (UID: \"347bfee3-5a7a-48b1-a227-0689c2790b5c\") " pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" Jul 16 17:22:17.711692 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:17.711651 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/347bfee3-5a7a-48b1-a227-0689c2790b5c-credential-socket\") pod \"e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54\" (UID: \"347bfee3-5a7a-48b1-a227-0689c2790b5c\") " pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" Jul 16 17:22:17.711692 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:17.711665 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/347bfee3-5a7a-48b1-a227-0689c2790b5c-istio-data\") pod \"e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54\" (UID: \"347bfee3-5a7a-48b1-a227-0689c2790b5c\") " pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" Jul 16 17:22:17.712004 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:17.711809 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/347bfee3-5a7a-48b1-a227-0689c2790b5c-istio-podinfo\") pod \"e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54\" (UID: \"347bfee3-5a7a-48b1-a227-0689c2790b5c\") " pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" Jul 16 17:22:17.712004 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:17.711867 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/347bfee3-5a7a-48b1-a227-0689c2790b5c-istio-token\") pod \"e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54\" (UID: \"347bfee3-5a7a-48b1-a227-0689c2790b5c\") " pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" Jul 16 17:22:17.712004 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:17.711950 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/347bfee3-5a7a-48b1-a227-0689c2790b5c-istiod-ca-cert\") pod \"e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54\" (UID: \"347bfee3-5a7a-48b1-a227-0689c2790b5c\") " pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" Jul 16 17:22:17.712168 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:17.712036 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/347bfee3-5a7a-48b1-a227-0689c2790b5c-credential-socket\") pod \"e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54\" (UID: \"347bfee3-5a7a-48b1-a227-0689c2790b5c\") " pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" Jul 16 17:22:17.712372 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:17.712352 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/347bfee3-5a7a-48b1-a227-0689c2790b5c-istio-data\") pod \"e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54\" (UID: \"347bfee3-5a7a-48b1-a227-0689c2790b5c\") " pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" Jul 16 17:22:17.712372 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:17.712365 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/347bfee3-5a7a-48b1-a227-0689c2790b5c-workload-certs\") pod \"e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54\" (UID: \"347bfee3-5a7a-48b1-a227-0689c2790b5c\") " pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" Jul 16 17:22:17.712520 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:17.712350 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/347bfee3-5a7a-48b1-a227-0689c2790b5c-workload-socket\") pod \"e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54\" (UID: \"347bfee3-5a7a-48b1-a227-0689c2790b5c\") " pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" Jul 16 17:22:17.712798 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:17.712773 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/347bfee3-5a7a-48b1-a227-0689c2790b5c-istiod-ca-cert\") pod \"e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54\" (UID: \"347bfee3-5a7a-48b1-a227-0689c2790b5c\") " pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" Jul 16 17:22:17.714215 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:17.714200 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/347bfee3-5a7a-48b1-a227-0689c2790b5c-istio-envoy\") pod \"e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54\" (UID: \"347bfee3-5a7a-48b1-a227-0689c2790b5c\") " pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" Jul 16 17:22:17.714510 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:17.714490 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/347bfee3-5a7a-48b1-a227-0689c2790b5c-istio-podinfo\") pod \"e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54\" (UID: \"347bfee3-5a7a-48b1-a227-0689c2790b5c\") " pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" Jul 16 17:22:17.725824 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:17.725797 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/347bfee3-5a7a-48b1-a227-0689c2790b5c-istio-token\") pod \"e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54\" (UID: \"347bfee3-5a7a-48b1-a227-0689c2790b5c\") " pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" Jul 16 17:22:17.731418 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:17.731397 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpd5l\" (UniqueName: \"kubernetes.io/projected/347bfee3-5a7a-48b1-a227-0689c2790b5c-kube-api-access-tpd5l\") pod \"e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54\" (UID: \"347bfee3-5a7a-48b1-a227-0689c2790b5c\") " pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" Jul 16 17:22:17.860618 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:17.860560 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" Jul 16 17:22:18.007994 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:18.007969 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54"] Jul 16 17:22:18.009744 ip-10-0-128-50 kubenswrapper[2571]: W0716 17:22:18.009702 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod347bfee3_5a7a_48b1_a227_0689c2790b5c.slice/crio-063ace5bd9e757dee6a037d52dbf273d1ed34a6d3a483ee558453bf3b4730420 WatchSource:0}: Error finding container 063ace5bd9e757dee6a037d52dbf273d1ed34a6d3a483ee558453bf3b4730420: Status 404 returned error can't find the container with id 063ace5bd9e757dee6a037d52dbf273d1ed34a6d3a483ee558453bf3b4730420 Jul 16 17:22:18.011618 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:18.011588 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 17:22:18.011695 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:18.011666 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 17:22:18.011758 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:18.011694 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 17:22:18.366536 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:18.366503 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" event={"ID":"347bfee3-5a7a-48b1-a227-0689c2790b5c","Type":"ContainerStarted","Data":"bb3aca147dc981cb17c7c8802f1e0cb50373dfbdaecf80bc0c2129b982470e84"} Jul 16 17:22:18.366710 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:18.366555 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" event={"ID":"347bfee3-5a7a-48b1-a227-0689c2790b5c","Type":"ContainerStarted","Data":"063ace5bd9e757dee6a037d52dbf273d1ed34a6d3a483ee558453bf3b4730420"} Jul 16 17:22:18.399464 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:18.399408 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" podStartSLOduration=1.399394121 podStartE2EDuration="1.399394121s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:22:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:22:18.397461735 +0000 UTC m=+1890.139037032" watchObservedRunningTime="2026-07-16 17:22:18.399394121 +0000 UTC m=+1890.140969417" Jul 16 17:22:18.861670 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:18.861642 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" Jul 16 17:22:19.866343 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:19.866313 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" Jul 16 17:22:20.375558 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:20.375527 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" Jul 16 17:22:20.376446 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:20.376428 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" Jul 16 17:22:36.132101 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:36.131096 2571 patch_prober.go:28] interesting pod/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": read tcp 10.132.0.2:34758->10.132.0.54:15021: read: connection reset by peer" start-of-body= Jul 16 17:22:36.132101 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:36.131170 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" podUID="347bfee3-5a7a-48b1-a227-0689c2790b5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": read tcp 10.132.0.2:34758->10.132.0.54:15021: read: connection reset by peer" Jul 16 17:22:36.461981 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:36.461953 2571 generic.go:356] "Generic (PLEG): container finished" podID="347bfee3-5a7a-48b1-a227-0689c2790b5c" containerID="bb3aca147dc981cb17c7c8802f1e0cb50373dfbdaecf80bc0c2129b982470e84" exitCode=0 Jul 16 17:22:36.462142 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:36.462032 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" event={"ID":"347bfee3-5a7a-48b1-a227-0689c2790b5c","Type":"ContainerDied","Data":"bb3aca147dc981cb17c7c8802f1e0cb50373dfbdaecf80bc0c2129b982470e84"} Jul 16 17:22:36.462421 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:36.462402 2571 scope.go:117] "RemoveContainer" containerID="bb3aca147dc981cb17c7c8802f1e0cb50373dfbdaecf80bc0c2129b982470e84" Jul 16 17:22:36.463341 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:36.463315 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 17:22:36.463390 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:36.463379 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 17:22:36.463425 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:36.463410 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 17:22:37.469396 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:37.469354 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" event={"ID":"347bfee3-5a7a-48b1-a227-0689c2790b5c","Type":"ContainerStarted","Data":"606d5e992a28f1820f3035c6561e69d9d46a9e2e1a263e20d4948c25339cab70"} Jul 16 17:22:37.860752 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:37.860654 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" Jul 16 17:22:37.860950 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:37.860928 2571 patch_prober.go:28] interesting pod/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 16 17:22:37.861013 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:37.860993 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" podUID="347bfee3-5a7a-48b1-a227-0689c2790b5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 16 17:22:38.861888 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:38.861852 2571 patch_prober.go:28] interesting pod/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 16 17:22:38.862265 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:38.861905 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" podUID="347bfee3-5a7a-48b1-a227-0689c2790b5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 16 17:22:39.861421 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:39.861389 2571 patch_prober.go:28] interesting pod/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 16 17:22:39.861591 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:39.861436 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" podUID="347bfee3-5a7a-48b1-a227-0689c2790b5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 16 17:22:40.861407 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:40.861376 2571 patch_prober.go:28] interesting pod/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 16 17:22:40.861807 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:40.861428 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" podUID="347bfee3-5a7a-48b1-a227-0689c2790b5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 16 17:22:41.864642 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:41.864615 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" Jul 16 17:22:41.865016 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:41.864948 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" Jul 16 17:22:41.865615 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:41.865593 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" Jul 16 17:22:49.874477 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:49.874444 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-7b9f8f/test-model-7b9f8f-kserve-5cbcf48785-8rjrz"] Jul 16 17:22:49.874985 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:49.874832 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="5410b4e8-f4d2-439e-a5a8-970a04a4780e" containerName="istio-proxy" Jul 16 17:22:49.877897 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:49.877879 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-7b9f8f/test-model-7b9f8f-kserve-5cbcf48785-8rjrz" Jul 16 17:22:49.882643 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:49.882625 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-7b9f8f\"/\"default-dockercfg-b5skl\"" Jul 16 17:22:49.884458 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:49.884097 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-7b9f8f\"/\"kube-root-ca.crt\"" Jul 16 17:22:49.884458 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:49.884121 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-7b9f8f\"/\"test-model-7b9f8f-kserve-self-signed-certs\"" Jul 16 17:22:49.884458 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:49.884151 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-7b9f8f\"/\"openshift-service-ca.crt\"" Jul 16 17:22:49.894085 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:49.894066 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-7b9f8f/test-model-7b9f8f-kserve-5cbcf48785-8rjrz"] Jul 16 17:22:49.976664 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:49.976633 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2723961e-9d5d-4884-8ff0-0e96821582e2-model-cache\") pod \"test-model-7b9f8f-kserve-5cbcf48785-8rjrz\" (UID: \"2723961e-9d5d-4884-8ff0-0e96821582e2\") " pod="ai-tenant-e2e-inf-a-7b9f8f/test-model-7b9f8f-kserve-5cbcf48785-8rjrz" Jul 16 17:22:49.976664 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:49.976659 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2723961e-9d5d-4884-8ff0-0e96821582e2-tls-certs\") pod \"test-model-7b9f8f-kserve-5cbcf48785-8rjrz\" (UID: \"2723961e-9d5d-4884-8ff0-0e96821582e2\") " pod="ai-tenant-e2e-inf-a-7b9f8f/test-model-7b9f8f-kserve-5cbcf48785-8rjrz" Jul 16 17:22:49.976879 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:49.976693 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2723961e-9d5d-4884-8ff0-0e96821582e2-dshm\") pod \"test-model-7b9f8f-kserve-5cbcf48785-8rjrz\" (UID: \"2723961e-9d5d-4884-8ff0-0e96821582e2\") " pod="ai-tenant-e2e-inf-a-7b9f8f/test-model-7b9f8f-kserve-5cbcf48785-8rjrz" Jul 16 17:22:49.976879 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:49.976788 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2723961e-9d5d-4884-8ff0-0e96821582e2-home\") pod \"test-model-7b9f8f-kserve-5cbcf48785-8rjrz\" (UID: \"2723961e-9d5d-4884-8ff0-0e96821582e2\") " pod="ai-tenant-e2e-inf-a-7b9f8f/test-model-7b9f8f-kserve-5cbcf48785-8rjrz" Jul 16 17:22:49.976879 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:49.976817 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2723961e-9d5d-4884-8ff0-0e96821582e2-tmp-dir\") pod \"test-model-7b9f8f-kserve-5cbcf48785-8rjrz\" (UID: \"2723961e-9d5d-4884-8ff0-0e96821582e2\") " pod="ai-tenant-e2e-inf-a-7b9f8f/test-model-7b9f8f-kserve-5cbcf48785-8rjrz" Jul 16 17:22:49.976879 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:49.976833 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsh7q\" (UniqueName: \"kubernetes.io/projected/2723961e-9d5d-4884-8ff0-0e96821582e2-kube-api-access-wsh7q\") pod \"test-model-7b9f8f-kserve-5cbcf48785-8rjrz\" (UID: \"2723961e-9d5d-4884-8ff0-0e96821582e2\") " pod="ai-tenant-e2e-inf-a-7b9f8f/test-model-7b9f8f-kserve-5cbcf48785-8rjrz" Jul 16 17:22:50.077393 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:50.077369 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2723961e-9d5d-4884-8ff0-0e96821582e2-tmp-dir\") pod \"test-model-7b9f8f-kserve-5cbcf48785-8rjrz\" (UID: \"2723961e-9d5d-4884-8ff0-0e96821582e2\") " pod="ai-tenant-e2e-inf-a-7b9f8f/test-model-7b9f8f-kserve-5cbcf48785-8rjrz" Jul 16 17:22:50.077545 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:50.077397 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsh7q\" (UniqueName: \"kubernetes.io/projected/2723961e-9d5d-4884-8ff0-0e96821582e2-kube-api-access-wsh7q\") pod \"test-model-7b9f8f-kserve-5cbcf48785-8rjrz\" (UID: \"2723961e-9d5d-4884-8ff0-0e96821582e2\") " pod="ai-tenant-e2e-inf-a-7b9f8f/test-model-7b9f8f-kserve-5cbcf48785-8rjrz" Jul 16 17:22:50.077545 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:50.077438 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2723961e-9d5d-4884-8ff0-0e96821582e2-model-cache\") pod \"test-model-7b9f8f-kserve-5cbcf48785-8rjrz\" (UID: \"2723961e-9d5d-4884-8ff0-0e96821582e2\") " pod="ai-tenant-e2e-inf-a-7b9f8f/test-model-7b9f8f-kserve-5cbcf48785-8rjrz" Jul 16 17:22:50.077545 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:50.077453 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2723961e-9d5d-4884-8ff0-0e96821582e2-tls-certs\") pod \"test-model-7b9f8f-kserve-5cbcf48785-8rjrz\" (UID: \"2723961e-9d5d-4884-8ff0-0e96821582e2\") " pod="ai-tenant-e2e-inf-a-7b9f8f/test-model-7b9f8f-kserve-5cbcf48785-8rjrz" Jul 16 17:22:50.077545 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:50.077487 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2723961e-9d5d-4884-8ff0-0e96821582e2-dshm\") pod \"test-model-7b9f8f-kserve-5cbcf48785-8rjrz\" (UID: \"2723961e-9d5d-4884-8ff0-0e96821582e2\") " pod="ai-tenant-e2e-inf-a-7b9f8f/test-model-7b9f8f-kserve-5cbcf48785-8rjrz" Jul 16 17:22:50.077545 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:50.077518 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2723961e-9d5d-4884-8ff0-0e96821582e2-home\") pod \"test-model-7b9f8f-kserve-5cbcf48785-8rjrz\" (UID: \"2723961e-9d5d-4884-8ff0-0e96821582e2\") " pod="ai-tenant-e2e-inf-a-7b9f8f/test-model-7b9f8f-kserve-5cbcf48785-8rjrz" Jul 16 17:22:50.077871 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:50.077837 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2723961e-9d5d-4884-8ff0-0e96821582e2-tmp-dir\") pod \"test-model-7b9f8f-kserve-5cbcf48785-8rjrz\" (UID: \"2723961e-9d5d-4884-8ff0-0e96821582e2\") " pod="ai-tenant-e2e-inf-a-7b9f8f/test-model-7b9f8f-kserve-5cbcf48785-8rjrz" Jul 16 17:22:50.077871 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:50.077858 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2723961e-9d5d-4884-8ff0-0e96821582e2-model-cache\") pod \"test-model-7b9f8f-kserve-5cbcf48785-8rjrz\" (UID: \"2723961e-9d5d-4884-8ff0-0e96821582e2\") " pod="ai-tenant-e2e-inf-a-7b9f8f/test-model-7b9f8f-kserve-5cbcf48785-8rjrz" Jul 16 17:22:50.077985 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:50.077944 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2723961e-9d5d-4884-8ff0-0e96821582e2-home\") pod \"test-model-7b9f8f-kserve-5cbcf48785-8rjrz\" (UID: \"2723961e-9d5d-4884-8ff0-0e96821582e2\") " pod="ai-tenant-e2e-inf-a-7b9f8f/test-model-7b9f8f-kserve-5cbcf48785-8rjrz" Jul 16 17:22:50.079604 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:50.079587 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2723961e-9d5d-4884-8ff0-0e96821582e2-dshm\") pod \"test-model-7b9f8f-kserve-5cbcf48785-8rjrz\" (UID: \"2723961e-9d5d-4884-8ff0-0e96821582e2\") " pod="ai-tenant-e2e-inf-a-7b9f8f/test-model-7b9f8f-kserve-5cbcf48785-8rjrz" Jul 16 17:22:50.079788 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:50.079771 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2723961e-9d5d-4884-8ff0-0e96821582e2-tls-certs\") pod \"test-model-7b9f8f-kserve-5cbcf48785-8rjrz\" (UID: \"2723961e-9d5d-4884-8ff0-0e96821582e2\") " pod="ai-tenant-e2e-inf-a-7b9f8f/test-model-7b9f8f-kserve-5cbcf48785-8rjrz" Jul 16 17:22:50.088279 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:50.088260 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsh7q\" (UniqueName: \"kubernetes.io/projected/2723961e-9d5d-4884-8ff0-0e96821582e2-kube-api-access-wsh7q\") pod \"test-model-7b9f8f-kserve-5cbcf48785-8rjrz\" (UID: \"2723961e-9d5d-4884-8ff0-0e96821582e2\") " pod="ai-tenant-e2e-inf-a-7b9f8f/test-model-7b9f8f-kserve-5cbcf48785-8rjrz" Jul 16 17:22:50.187019 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:50.186984 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-7b9f8f/test-model-7b9f8f-kserve-5cbcf48785-8rjrz" Jul 16 17:22:50.331877 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:50.331853 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-7b9f8f/test-model-7b9f8f-kserve-5cbcf48785-8rjrz"] Jul 16 17:22:50.333368 ip-10-0-128-50 kubenswrapper[2571]: W0716 17:22:50.333346 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2723961e_9d5d_4884_8ff0_0e96821582e2.slice/crio-582ee4f27589adaf8cf8818f6d9c40262937f5e62985a6d8d24d2ec11ea5ca6c WatchSource:0}: Error finding container 582ee4f27589adaf8cf8818f6d9c40262937f5e62985a6d8d24d2ec11ea5ca6c: Status 404 returned error can't find the container with id 582ee4f27589adaf8cf8818f6d9c40262937f5e62985a6d8d24d2ec11ea5ca6c Jul 16 17:22:50.536528 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:50.536452 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-7b9f8f/test-model-7b9f8f-kserve-5cbcf48785-8rjrz" event={"ID":"2723961e-9d5d-4884-8ff0-0e96821582e2","Type":"ContainerStarted","Data":"1507ed626d236dd2d79776d38c29a5ab9d668106ce2b7f8699a99240fefc18fe"} Jul 16 17:22:50.536528 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:50.536492 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-7b9f8f/test-model-7b9f8f-kserve-5cbcf48785-8rjrz" event={"ID":"2723961e-9d5d-4884-8ff0-0e96821582e2","Type":"ContainerStarted","Data":"582ee4f27589adaf8cf8818f6d9c40262937f5e62985a6d8d24d2ec11ea5ca6c"} Jul 16 17:22:50.559937 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:22:50.559891 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-7b9f8f/test-model-7b9f8f-kserve-5cbcf48785-8rjrz" podStartSLOduration=1.559876842 podStartE2EDuration="1.559876842s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:22:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:22:50.558525287 +0000 UTC m=+1922.300100764" watchObservedRunningTime="2026-07-16 17:22:50.559876842 +0000 UTC m=+1922.301452185" Jul 16 17:23:00.187995 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:23:00.187957 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-7b9f8f/test-model-7b9f8f-kserve-5cbcf48785-8rjrz" Jul 16 17:23:00.188437 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:23:00.188007 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-7b9f8f/test-model-7b9f8f-kserve-5cbcf48785-8rjrz" Jul 16 17:23:00.202178 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:23:00.202153 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-7b9f8f/test-model-7b9f8f-kserve-5cbcf48785-8rjrz" Jul 16 17:23:00.599757 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:23:00.599656 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-7b9f8f/test-model-7b9f8f-kserve-5cbcf48785-8rjrz" Jul 16 17:24:16.698967 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:24:16.698894 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-7b9f8f/test-model-7b9f8f-kserve-5cbcf48785-8rjrz"] Jul 16 17:24:31.718747 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:24:31.718682 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-7b9f8f/test-model-7b9f8f-kserve-5cbcf48785-8rjrz" podUID="2723961e-9d5d-4884-8ff0-0e96821582e2" containerName="main" containerID="cri-o://1507ed626d236dd2d79776d38c29a5ab9d668106ce2b7f8699a99240fefc18fe" gracePeriod=45 Jul 16 17:24:31.967189 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:24:31.967163 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-7b9f8f/test-model-7b9f8f-kserve-5cbcf48785-8rjrz" Jul 16 17:24:32.048806 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:24:32.048747 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2723961e-9d5d-4884-8ff0-0e96821582e2-tmp-dir\") pod \"2723961e-9d5d-4884-8ff0-0e96821582e2\" (UID: \"2723961e-9d5d-4884-8ff0-0e96821582e2\") " Jul 16 17:24:32.048806 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:24:32.048799 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2723961e-9d5d-4884-8ff0-0e96821582e2-model-cache\") pod \"2723961e-9d5d-4884-8ff0-0e96821582e2\" (UID: \"2723961e-9d5d-4884-8ff0-0e96821582e2\") " Jul 16 17:24:32.048960 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:24:32.048826 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wsh7q\" (UniqueName: \"kubernetes.io/projected/2723961e-9d5d-4884-8ff0-0e96821582e2-kube-api-access-wsh7q\") pod \"2723961e-9d5d-4884-8ff0-0e96821582e2\" (UID: \"2723961e-9d5d-4884-8ff0-0e96821582e2\") " Jul 16 17:24:32.048960 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:24:32.048847 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2723961e-9d5d-4884-8ff0-0e96821582e2-dshm\") pod \"2723961e-9d5d-4884-8ff0-0e96821582e2\" (UID: \"2723961e-9d5d-4884-8ff0-0e96821582e2\") " Jul 16 17:24:32.048960 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:24:32.048866 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2723961e-9d5d-4884-8ff0-0e96821582e2-home\") pod \"2723961e-9d5d-4884-8ff0-0e96821582e2\" (UID: \"2723961e-9d5d-4884-8ff0-0e96821582e2\") " Jul 16 17:24:32.048960 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:24:32.048887 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2723961e-9d5d-4884-8ff0-0e96821582e2-tls-certs\") pod \"2723961e-9d5d-4884-8ff0-0e96821582e2\" (UID: \"2723961e-9d5d-4884-8ff0-0e96821582e2\") " Jul 16 17:24:32.049150 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:24:32.048973 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2723961e-9d5d-4884-8ff0-0e96821582e2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2723961e-9d5d-4884-8ff0-0e96821582e2" (UID: "2723961e-9d5d-4884-8ff0-0e96821582e2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:24:32.049150 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:24:32.049103 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2723961e-9d5d-4884-8ff0-0e96821582e2-model-cache" (OuterVolumeSpecName: "model-cache") pod "2723961e-9d5d-4884-8ff0-0e96821582e2" (UID: "2723961e-9d5d-4884-8ff0-0e96821582e2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:24:32.049272 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:24:32.049140 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2723961e-9d5d-4884-8ff0-0e96821582e2-home" (OuterVolumeSpecName: "home") pod "2723961e-9d5d-4884-8ff0-0e96821582e2" (UID: "2723961e-9d5d-4884-8ff0-0e96821582e2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:24:32.049272 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:24:32.049208 2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2723961e-9d5d-4884-8ff0-0e96821582e2-home\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:24:32.049272 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:24:32.049242 2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2723961e-9d5d-4884-8ff0-0e96821582e2-tmp-dir\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:24:32.049272 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:24:32.049251 2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2723961e-9d5d-4884-8ff0-0e96821582e2-model-cache\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:24:32.051010 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:24:32.050983 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2723961e-9d5d-4884-8ff0-0e96821582e2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2723961e-9d5d-4884-8ff0-0e96821582e2" (UID: "2723961e-9d5d-4884-8ff0-0e96821582e2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:24:32.051094 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:24:32.051039 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2723961e-9d5d-4884-8ff0-0e96821582e2-kube-api-access-wsh7q" (OuterVolumeSpecName: "kube-api-access-wsh7q") pod "2723961e-9d5d-4884-8ff0-0e96821582e2" (UID: "2723961e-9d5d-4884-8ff0-0e96821582e2"). InnerVolumeSpecName "kube-api-access-wsh7q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:24:32.051094 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:24:32.051079 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2723961e-9d5d-4884-8ff0-0e96821582e2-dshm" (OuterVolumeSpecName: "dshm") pod "2723961e-9d5d-4884-8ff0-0e96821582e2" (UID: "2723961e-9d5d-4884-8ff0-0e96821582e2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:24:32.060815 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:24:32.060789 2571 generic.go:356] "Generic (PLEG): container finished" podID="2723961e-9d5d-4884-8ff0-0e96821582e2" containerID="1507ed626d236dd2d79776d38c29a5ab9d668106ce2b7f8699a99240fefc18fe" exitCode=0 Jul 16 17:24:32.060898 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:24:32.060835 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-7b9f8f/test-model-7b9f8f-kserve-5cbcf48785-8rjrz" Jul 16 17:24:32.060898 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:24:32.060846 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-7b9f8f/test-model-7b9f8f-kserve-5cbcf48785-8rjrz" event={"ID":"2723961e-9d5d-4884-8ff0-0e96821582e2","Type":"ContainerDied","Data":"1507ed626d236dd2d79776d38c29a5ab9d668106ce2b7f8699a99240fefc18fe"} Jul 16 17:24:32.060898 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:24:32.060884 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-7b9f8f/test-model-7b9f8f-kserve-5cbcf48785-8rjrz" event={"ID":"2723961e-9d5d-4884-8ff0-0e96821582e2","Type":"ContainerDied","Data":"582ee4f27589adaf8cf8818f6d9c40262937f5e62985a6d8d24d2ec11ea5ca6c"} Jul 16 17:24:32.060996 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:24:32.060905 2571 scope.go:117] "RemoveContainer" containerID="1507ed626d236dd2d79776d38c29a5ab9d668106ce2b7f8699a99240fefc18fe" Jul 16 17:24:32.069489 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:24:32.069474 2571 scope.go:117] "RemoveContainer" containerID="1507ed626d236dd2d79776d38c29a5ab9d668106ce2b7f8699a99240fefc18fe" Jul 16 17:24:32.069745 ip-10-0-128-50 kubenswrapper[2571]: E0716 17:24:32.069709 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1507ed626d236dd2d79776d38c29a5ab9d668106ce2b7f8699a99240fefc18fe\": container with ID starting with 1507ed626d236dd2d79776d38c29a5ab9d668106ce2b7f8699a99240fefc18fe not found: ID does not exist" containerID="1507ed626d236dd2d79776d38c29a5ab9d668106ce2b7f8699a99240fefc18fe" Jul 16 17:24:32.069789 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:24:32.069754 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1507ed626d236dd2d79776d38c29a5ab9d668106ce2b7f8699a99240fefc18fe"} err="failed to get container status \"1507ed626d236dd2d79776d38c29a5ab9d668106ce2b7f8699a99240fefc18fe\": rpc error: code = NotFound desc = could not find container \"1507ed626d236dd2d79776d38c29a5ab9d668106ce2b7f8699a99240fefc18fe\": container with ID starting with 1507ed626d236dd2d79776d38c29a5ab9d668106ce2b7f8699a99240fefc18fe not found: ID does not exist" Jul 16 17:24:32.089306 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:24:32.089282 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-7b9f8f/test-model-7b9f8f-kserve-5cbcf48785-8rjrz"] Jul 16 17:24:32.093558 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:24:32.093538 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-7b9f8f/test-model-7b9f8f-kserve-5cbcf48785-8rjrz"] Jul 16 17:24:32.149825 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:24:32.149805 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wsh7q\" (UniqueName: \"kubernetes.io/projected/2723961e-9d5d-4884-8ff0-0e96821582e2-kube-api-access-wsh7q\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:24:32.149825 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:24:32.149826 2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2723961e-9d5d-4884-8ff0-0e96821582e2-dshm\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:24:32.149943 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:24:32.149836 2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2723961e-9d5d-4884-8ff0-0e96821582e2-tls-certs\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:24:32.776918 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:24:32.776891 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2723961e-9d5d-4884-8ff0-0e96821582e2" path="/var/lib/kubelet/pods/2723961e-9d5d-4884-8ff0-0e96821582e2/volumes" Jul 16 17:25:11.705346 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:11.705268 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54"] Jul 16 17:25:11.706050 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:11.705464 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" podUID="347bfee3-5a7a-48b1-a227-0689c2790b5c" containerName="istio-proxy" containerID="cri-o://606d5e992a28f1820f3035c6561e69d9d46a9e2e1a263e20d4948c25339cab70" gracePeriod=30 Jul 16 17:25:11.866644 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:11.866616 2571 patch_prober.go:28] interesting pod/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 16 17:25:11.866830 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:11.866665 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" podUID="347bfee3-5a7a-48b1-a227-0689c2790b5c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 17:25:16.963925 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:16.963865 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" Jul 16 17:25:17.086104 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:17.086081 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tpd5l\" (UniqueName: \"kubernetes.io/projected/347bfee3-5a7a-48b1-a227-0689c2790b5c-kube-api-access-tpd5l\") pod \"347bfee3-5a7a-48b1-a227-0689c2790b5c\" (UID: \"347bfee3-5a7a-48b1-a227-0689c2790b5c\") " Jul 16 17:25:17.086263 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:17.086119 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/347bfee3-5a7a-48b1-a227-0689c2790b5c-istio-podinfo\") pod \"347bfee3-5a7a-48b1-a227-0689c2790b5c\" (UID: \"347bfee3-5a7a-48b1-a227-0689c2790b5c\") " Jul 16 17:25:17.086263 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:17.086148 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/347bfee3-5a7a-48b1-a227-0689c2790b5c-istio-envoy\") pod \"347bfee3-5a7a-48b1-a227-0689c2790b5c\" (UID: \"347bfee3-5a7a-48b1-a227-0689c2790b5c\") " Jul 16 17:25:17.086263 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:17.086200 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/347bfee3-5a7a-48b1-a227-0689c2790b5c-workload-certs\") pod \"347bfee3-5a7a-48b1-a227-0689c2790b5c\" (UID: \"347bfee3-5a7a-48b1-a227-0689c2790b5c\") " Jul 16 17:25:17.086263 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:17.086224 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/347bfee3-5a7a-48b1-a227-0689c2790b5c-istio-data\") pod \"347bfee3-5a7a-48b1-a227-0689c2790b5c\" (UID: \"347bfee3-5a7a-48b1-a227-0689c2790b5c\") " Jul 16 17:25:17.086495 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:17.086274 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/347bfee3-5a7a-48b1-a227-0689c2790b5c-istio-token\") pod \"347bfee3-5a7a-48b1-a227-0689c2790b5c\" (UID: \"347bfee3-5a7a-48b1-a227-0689c2790b5c\") " Jul 16 17:25:17.086495 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:17.086327 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/347bfee3-5a7a-48b1-a227-0689c2790b5c-istiod-ca-cert\") pod \"347bfee3-5a7a-48b1-a227-0689c2790b5c\" (UID: \"347bfee3-5a7a-48b1-a227-0689c2790b5c\") " Jul 16 17:25:17.086495 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:17.086377 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/347bfee3-5a7a-48b1-a227-0689c2790b5c-workload-socket\") pod \"347bfee3-5a7a-48b1-a227-0689c2790b5c\" (UID: \"347bfee3-5a7a-48b1-a227-0689c2790b5c\") " Jul 16 17:25:17.086495 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:17.086460 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/347bfee3-5a7a-48b1-a227-0689c2790b5c-credential-socket\") pod \"347bfee3-5a7a-48b1-a227-0689c2790b5c\" (UID: \"347bfee3-5a7a-48b1-a227-0689c2790b5c\") " Jul 16 17:25:17.086495 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:17.086462 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/347bfee3-5a7a-48b1-a227-0689c2790b5c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "347bfee3-5a7a-48b1-a227-0689c2790b5c" (UID: "347bfee3-5a7a-48b1-a227-0689c2790b5c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:25:17.086754 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:17.086690 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/347bfee3-5a7a-48b1-a227-0689c2790b5c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "347bfee3-5a7a-48b1-a227-0689c2790b5c" (UID: "347bfee3-5a7a-48b1-a227-0689c2790b5c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:25:17.086754 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:17.086706 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/347bfee3-5a7a-48b1-a227-0689c2790b5c-workload-certs\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:25:17.086880 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:17.086768 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/347bfee3-5a7a-48b1-a227-0689c2790b5c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "347bfee3-5a7a-48b1-a227-0689c2790b5c" (UID: "347bfee3-5a7a-48b1-a227-0689c2790b5c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:25:17.086965 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:17.086947 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/347bfee3-5a7a-48b1-a227-0689c2790b5c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "347bfee3-5a7a-48b1-a227-0689c2790b5c" (UID: "347bfee3-5a7a-48b1-a227-0689c2790b5c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:25:17.087170 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:17.087138 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/347bfee3-5a7a-48b1-a227-0689c2790b5c-istio-data" (OuterVolumeSpecName: "istio-data") pod "347bfee3-5a7a-48b1-a227-0689c2790b5c" (UID: "347bfee3-5a7a-48b1-a227-0689c2790b5c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:25:17.088266 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:17.088242 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/347bfee3-5a7a-48b1-a227-0689c2790b5c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "347bfee3-5a7a-48b1-a227-0689c2790b5c" (UID: "347bfee3-5a7a-48b1-a227-0689c2790b5c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:25:17.088345 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:17.088280 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/347bfee3-5a7a-48b1-a227-0689c2790b5c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "347bfee3-5a7a-48b1-a227-0689c2790b5c" (UID: "347bfee3-5a7a-48b1-a227-0689c2790b5c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 17:25:17.088385 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:17.088334 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/347bfee3-5a7a-48b1-a227-0689c2790b5c-istio-token" (OuterVolumeSpecName: "istio-token") pod "347bfee3-5a7a-48b1-a227-0689c2790b5c" (UID: "347bfee3-5a7a-48b1-a227-0689c2790b5c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:25:17.088539 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:17.088519 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/347bfee3-5a7a-48b1-a227-0689c2790b5c-kube-api-access-tpd5l" (OuterVolumeSpecName: "kube-api-access-tpd5l") pod "347bfee3-5a7a-48b1-a227-0689c2790b5c" (UID: "347bfee3-5a7a-48b1-a227-0689c2790b5c"). InnerVolumeSpecName "kube-api-access-tpd5l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:25:17.187971 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:17.187948 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/347bfee3-5a7a-48b1-a227-0689c2790b5c-workload-socket\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:25:17.187971 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:17.187969 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/347bfee3-5a7a-48b1-a227-0689c2790b5c-credential-socket\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:25:17.188093 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:17.187979 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tpd5l\" (UniqueName: \"kubernetes.io/projected/347bfee3-5a7a-48b1-a227-0689c2790b5c-kube-api-access-tpd5l\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:25:17.188093 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:17.187987 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/347bfee3-5a7a-48b1-a227-0689c2790b5c-istio-podinfo\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:25:17.188093 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:17.187995 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/347bfee3-5a7a-48b1-a227-0689c2790b5c-istio-envoy\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:25:17.188093 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:17.188002 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/347bfee3-5a7a-48b1-a227-0689c2790b5c-istio-data\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:25:17.188093 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:17.188009 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/347bfee3-5a7a-48b1-a227-0689c2790b5c-istio-token\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:25:17.188093 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:17.188016 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/347bfee3-5a7a-48b1-a227-0689c2790b5c-istiod-ca-cert\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:25:17.288957 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:17.288904 2571 generic.go:356] "Generic (PLEG): container finished" podID="347bfee3-5a7a-48b1-a227-0689c2790b5c" containerID="606d5e992a28f1820f3035c6561e69d9d46a9e2e1a263e20d4948c25339cab70" exitCode=0 Jul 16 17:25:17.288957 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:17.288940 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" event={"ID":"347bfee3-5a7a-48b1-a227-0689c2790b5c","Type":"ContainerDied","Data":"606d5e992a28f1820f3035c6561e69d9d46a9e2e1a263e20d4948c25339cab70"} Jul 16 17:25:17.288957 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:17.288945 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" Jul 16 17:25:17.289092 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:17.288984 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54" event={"ID":"347bfee3-5a7a-48b1-a227-0689c2790b5c","Type":"ContainerDied","Data":"063ace5bd9e757dee6a037d52dbf273d1ed34a6d3a483ee558453bf3b4730420"} Jul 16 17:25:17.289092 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:17.289010 2571 scope.go:117] "RemoveContainer" containerID="606d5e992a28f1820f3035c6561e69d9d46a9e2e1a263e20d4948c25339cab70" Jul 16 17:25:17.297485 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:17.297469 2571 scope.go:117] "RemoveContainer" containerID="bb3aca147dc981cb17c7c8802f1e0cb50373dfbdaecf80bc0c2129b982470e84" Jul 16 17:25:17.305008 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:17.304991 2571 scope.go:117] "RemoveContainer" containerID="606d5e992a28f1820f3035c6561e69d9d46a9e2e1a263e20d4948c25339cab70" Jul 16 17:25:17.305251 ip-10-0-128-50 kubenswrapper[2571]: E0716 17:25:17.305233 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"606d5e992a28f1820f3035c6561e69d9d46a9e2e1a263e20d4948c25339cab70\": container with ID starting with 606d5e992a28f1820f3035c6561e69d9d46a9e2e1a263e20d4948c25339cab70 not found: ID does not exist" containerID="606d5e992a28f1820f3035c6561e69d9d46a9e2e1a263e20d4948c25339cab70" Jul 16 17:25:17.305323 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:17.305264 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"606d5e992a28f1820f3035c6561e69d9d46a9e2e1a263e20d4948c25339cab70"} err="failed to get container status \"606d5e992a28f1820f3035c6561e69d9d46a9e2e1a263e20d4948c25339cab70\": rpc error: code = NotFound desc = could not find container \"606d5e992a28f1820f3035c6561e69d9d46a9e2e1a263e20d4948c25339cab70\": container with ID starting with 606d5e992a28f1820f3035c6561e69d9d46a9e2e1a263e20d4948c25339cab70 not found: ID does not exist" Jul 16 17:25:17.305323 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:17.305287 2571 scope.go:117] "RemoveContainer" containerID="bb3aca147dc981cb17c7c8802f1e0cb50373dfbdaecf80bc0c2129b982470e84" Jul 16 17:25:17.305525 ip-10-0-128-50 kubenswrapper[2571]: E0716 17:25:17.305507 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb3aca147dc981cb17c7c8802f1e0cb50373dfbdaecf80bc0c2129b982470e84\": container with ID starting with bb3aca147dc981cb17c7c8802f1e0cb50373dfbdaecf80bc0c2129b982470e84 not found: ID does not exist" containerID="bb3aca147dc981cb17c7c8802f1e0cb50373dfbdaecf80bc0c2129b982470e84" Jul 16 17:25:17.305566 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:17.305531 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb3aca147dc981cb17c7c8802f1e0cb50373dfbdaecf80bc0c2129b982470e84"} err="failed to get container status \"bb3aca147dc981cb17c7c8802f1e0cb50373dfbdaecf80bc0c2129b982470e84\": rpc error: code = NotFound desc = could not find container \"bb3aca147dc981cb17c7c8802f1e0cb50373dfbdaecf80bc0c2129b982470e84\": container with ID starting with bb3aca147dc981cb17c7c8802f1e0cb50373dfbdaecf80bc0c2129b982470e84 not found: ID does not exist" Jul 16 17:25:17.340131 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:17.340108 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54"] Jul 16 17:25:17.356208 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:17.356187 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-7b9f8f-openshift-default-cc6dd845c-bnh54"] Jul 16 17:25:18.776479 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:18.776451 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="347bfee3-5a7a-48b1-a227-0689c2790b5c" path="/var/lib/kubelet/pods/347bfee3-5a7a-48b1-a227-0689c2790b5c/volumes" Jul 16 17:25:32.208895 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.208861 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl"] Jul 16 17:25:32.209266 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.209192 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="347bfee3-5a7a-48b1-a227-0689c2790b5c" containerName="istio-proxy" Jul 16 17:25:32.209266 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.209203 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="347bfee3-5a7a-48b1-a227-0689c2790b5c" containerName="istio-proxy" Jul 16 17:25:32.209266 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.209218 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="347bfee3-5a7a-48b1-a227-0689c2790b5c" containerName="istio-proxy" Jul 16 17:25:32.209266 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.209223 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="347bfee3-5a7a-48b1-a227-0689c2790b5c" containerName="istio-proxy" Jul 16 17:25:32.209266 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.209232 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2723961e-9d5d-4884-8ff0-0e96821582e2" containerName="main" Jul 16 17:25:32.209266 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.209237 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2723961e-9d5d-4884-8ff0-0e96821582e2" containerName="main" Jul 16 17:25:32.209486 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.209289 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="2723961e-9d5d-4884-8ff0-0e96821582e2" containerName="main" Jul 16 17:25:32.209486 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.209297 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="347bfee3-5a7a-48b1-a227-0689c2790b5c" containerName="istio-proxy" Jul 16 17:25:32.209486 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.209305 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="347bfee3-5a7a-48b1-a227-0689c2790b5c" containerName="istio-proxy" Jul 16 17:25:32.212392 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.212375 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:25:32.218559 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.218523 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 16 17:25:32.218701 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.218635 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-36dc65-openshift-default-dockercfg-kh7pr\"" Jul 16 17:25:32.252125 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.252097 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl"] Jul 16 17:25:32.399455 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.399426 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdnpg\" (UniqueName: \"kubernetes.io/projected/4b563d02-f2b6-4186-8c24-97584eced510-kube-api-access-sdnpg\") pod \"e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl\" (UID: \"4b563d02-f2b6-4186-8c24-97584eced510\") " pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:25:32.399586 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.399464 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b563d02-f2b6-4186-8c24-97584eced510-istio-data\") pod \"e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl\" (UID: \"4b563d02-f2b6-4186-8c24-97584eced510\") " pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:25:32.399586 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.399487 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b563d02-f2b6-4186-8c24-97584eced510-istio-token\") pod \"e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl\" (UID: \"4b563d02-f2b6-4186-8c24-97584eced510\") " pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:25:32.399586 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.399537 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b563d02-f2b6-4186-8c24-97584eced510-workload-socket\") pod \"e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl\" (UID: \"4b563d02-f2b6-4186-8c24-97584eced510\") " pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:25:32.399586 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.399556 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b563d02-f2b6-4186-8c24-97584eced510-istio-envoy\") pod \"e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl\" (UID: \"4b563d02-f2b6-4186-8c24-97584eced510\") " pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:25:32.399719 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.399586 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b563d02-f2b6-4186-8c24-97584eced510-workload-certs\") pod \"e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl\" (UID: \"4b563d02-f2b6-4186-8c24-97584eced510\") " pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:25:32.399719 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.399607 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b563d02-f2b6-4186-8c24-97584eced510-istio-podinfo\") pod \"e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl\" (UID: \"4b563d02-f2b6-4186-8c24-97584eced510\") " pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:25:32.399719 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.399623 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b563d02-f2b6-4186-8c24-97584eced510-istiod-ca-cert\") pod \"e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl\" (UID: \"4b563d02-f2b6-4186-8c24-97584eced510\") " pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:25:32.399719 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.399664 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b563d02-f2b6-4186-8c24-97584eced510-credential-socket\") pod \"e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl\" (UID: \"4b563d02-f2b6-4186-8c24-97584eced510\") " pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:25:32.500819 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.500746 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdnpg\" (UniqueName: \"kubernetes.io/projected/4b563d02-f2b6-4186-8c24-97584eced510-kube-api-access-sdnpg\") pod \"e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl\" (UID: \"4b563d02-f2b6-4186-8c24-97584eced510\") " pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:25:32.500819 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.500786 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b563d02-f2b6-4186-8c24-97584eced510-istio-data\") pod \"e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl\" (UID: \"4b563d02-f2b6-4186-8c24-97584eced510\") " pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:25:32.500819 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.500811 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b563d02-f2b6-4186-8c24-97584eced510-istio-token\") pod \"e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl\" (UID: \"4b563d02-f2b6-4186-8c24-97584eced510\") " pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:25:32.501075 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.500854 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b563d02-f2b6-4186-8c24-97584eced510-workload-socket\") pod \"e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl\" (UID: \"4b563d02-f2b6-4186-8c24-97584eced510\") " pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:25:32.501075 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.500882 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b563d02-f2b6-4186-8c24-97584eced510-istio-envoy\") pod \"e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl\" (UID: \"4b563d02-f2b6-4186-8c24-97584eced510\") " pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:25:32.501075 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.500916 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b563d02-f2b6-4186-8c24-97584eced510-workload-certs\") pod \"e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl\" (UID: \"4b563d02-f2b6-4186-8c24-97584eced510\") " pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:25:32.501075 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.500942 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b563d02-f2b6-4186-8c24-97584eced510-istio-podinfo\") pod \"e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl\" (UID: \"4b563d02-f2b6-4186-8c24-97584eced510\") " pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:25:32.501075 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.500966 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b563d02-f2b6-4186-8c24-97584eced510-istiod-ca-cert\") pod \"e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl\" (UID: \"4b563d02-f2b6-4186-8c24-97584eced510\") " pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:25:32.501075 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.501000 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b563d02-f2b6-4186-8c24-97584eced510-credential-socket\") pod \"e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl\" (UID: \"4b563d02-f2b6-4186-8c24-97584eced510\") " pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:25:32.501402 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.501161 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b563d02-f2b6-4186-8c24-97584eced510-istio-data\") pod \"e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl\" (UID: \"4b563d02-f2b6-4186-8c24-97584eced510\") " pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:25:32.501402 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.501294 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b563d02-f2b6-4186-8c24-97584eced510-workload-socket\") pod \"e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl\" (UID: \"4b563d02-f2b6-4186-8c24-97584eced510\") " pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:25:32.501402 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.501335 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b563d02-f2b6-4186-8c24-97584eced510-workload-certs\") pod \"e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl\" (UID: \"4b563d02-f2b6-4186-8c24-97584eced510\") " pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:25:32.501402 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.501355 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b563d02-f2b6-4186-8c24-97584eced510-credential-socket\") pod \"e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl\" (UID: \"4b563d02-f2b6-4186-8c24-97584eced510\") " pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:25:32.501967 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.501948 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b563d02-f2b6-4186-8c24-97584eced510-istiod-ca-cert\") pod \"e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl\" (UID: \"4b563d02-f2b6-4186-8c24-97584eced510\") " pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:25:32.503343 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.503324 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b563d02-f2b6-4186-8c24-97584eced510-istio-envoy\") pod \"e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl\" (UID: \"4b563d02-f2b6-4186-8c24-97584eced510\") " pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:25:32.503474 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.503457 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b563d02-f2b6-4186-8c24-97584eced510-istio-podinfo\") pod \"e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl\" (UID: \"4b563d02-f2b6-4186-8c24-97584eced510\") " pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:25:32.528536 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.528519 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b563d02-f2b6-4186-8c24-97584eced510-istio-token\") pod \"e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl\" (UID: \"4b563d02-f2b6-4186-8c24-97584eced510\") " pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:25:32.536919 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.536898 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdnpg\" (UniqueName: \"kubernetes.io/projected/4b563d02-f2b6-4186-8c24-97584eced510-kube-api-access-sdnpg\") pod \"e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl\" (UID: \"4b563d02-f2b6-4186-8c24-97584eced510\") " pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:25:32.821872 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.821801 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:25:32.999940 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:32.999906 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl"] Jul 16 17:25:33.001458 ip-10-0-128-50 kubenswrapper[2571]: W0716 17:25:33.001436 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b563d02_f2b6_4186_8c24_97584eced510.slice/crio-cb90e931513044cb711bfe51a4d9febe5b73159191f81f77c7bb2fc71c22e357 WatchSource:0}: Error finding container cb90e931513044cb711bfe51a4d9febe5b73159191f81f77c7bb2fc71c22e357: Status 404 returned error can't find the container with id cb90e931513044cb711bfe51a4d9febe5b73159191f81f77c7bb2fc71c22e357 Jul 16 17:25:33.003417 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:33.003384 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 17:25:33.003472 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:33.003453 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 17:25:33.003507 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:33.003488 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 17:25:33.375651 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:33.375621 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" event={"ID":"4b563d02-f2b6-4186-8c24-97584eced510","Type":"ContainerStarted","Data":"e8b52235a588dd0d117da16aec16709662c96492bb84d940401eb32a7bdb0270"} Jul 16 17:25:33.376133 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:33.375671 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" event={"ID":"4b563d02-f2b6-4186-8c24-97584eced510","Type":"ContainerStarted","Data":"cb90e931513044cb711bfe51a4d9febe5b73159191f81f77c7bb2fc71c22e357"} Jul 16 17:25:33.423296 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:33.423247 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" podStartSLOduration=1.4232325239999999 podStartE2EDuration="1.423232524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:25:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:25:33.421575433 +0000 UTC m=+2085.163150729" watchObservedRunningTime="2026-07-16 17:25:33.423232524 +0000 UTC m=+2085.164807821" Jul 16 17:25:33.822164 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:33.822126 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:25:34.826130 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:34.826104 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:25:35.386439 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:35.386402 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:25:35.387182 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:35.387163 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:25:48.904541 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:48.904467 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r8cn4_f637d961-672d-4aa2-a49a-fd021c2d4df2/console-operator/2.log" Jul 16 17:25:48.909746 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:48.909700 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r8cn4_f637d961-672d-4aa2-a49a-fd021c2d4df2/console-operator/2.log" Jul 16 17:25:48.913074 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:48.913054 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qd7bg_cfbe9f0b-b759-4064-9332-3b48c547c003/ovn-acl-logging/0.log" Jul 16 17:25:48.919081 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:48.919062 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qd7bg_cfbe9f0b-b759-4064-9332-3b48c547c003/ovn-acl-logging/0.log" Jul 16 17:25:49.462631 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:49.462601 2571 generic.go:356] "Generic (PLEG): container finished" podID="4b563d02-f2b6-4186-8c24-97584eced510" containerID="e8b52235a588dd0d117da16aec16709662c96492bb84d940401eb32a7bdb0270" exitCode=0 Jul 16 17:25:49.462811 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:49.462640 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" event={"ID":"4b563d02-f2b6-4186-8c24-97584eced510","Type":"ContainerDied","Data":"e8b52235a588dd0d117da16aec16709662c96492bb84d940401eb32a7bdb0270"} Jul 16 17:25:49.463044 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:49.463026 2571 scope.go:117] "RemoveContainer" containerID="e8b52235a588dd0d117da16aec16709662c96492bb84d940401eb32a7bdb0270" Jul 16 17:25:49.463993 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:49.463959 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 17:25:49.464115 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:49.464023 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 17:25:49.464115 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:49.464053 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 17:25:50.386573 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:50.386522 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:25:50.470187 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:50.470154 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" event={"ID":"4b563d02-f2b6-4186-8c24-97584eced510","Type":"ContainerStarted","Data":"55b1f2454b8ba8e621074d48c342aef8031d59fddfa4c2aab4f6ce75e8fc002f"} Jul 16 17:25:50.822655 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:50.822610 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:25:50.824177 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:50.824150 2571 patch_prober.go:28] interesting pod/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 16 17:25:50.824292 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:50.824200 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" podUID="4b563d02-f2b6-4186-8c24-97584eced510" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 16 17:25:51.823076 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:51.823032 2571 patch_prober.go:28] interesting pod/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 16 17:25:51.823463 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:51.823121 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" podUID="4b563d02-f2b6-4186-8c24-97584eced510" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 16 17:25:52.822386 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:52.822349 2571 patch_prober.go:28] interesting pod/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 16 17:25:52.822584 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:52.822417 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" podUID="4b563d02-f2b6-4186-8c24-97584eced510" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 16 17:25:53.822199 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:53.822162 2571 patch_prober.go:28] interesting pod/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 16 17:25:53.822683 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:53.822224 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" podUID="4b563d02-f2b6-4186-8c24-97584eced510" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 16 17:25:54.826051 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:54.826023 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:25:54.826450 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:54.826368 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:25:54.827061 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:25:54.827043 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:26:17.547027 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:17.546994 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl"] Jul 16 17:26:17.547497 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:17.547188 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" podUID="4b563d02-f2b6-4186-8c24-97584eced510" containerName="istio-proxy" containerID="cri-o://55b1f2454b8ba8e621074d48c342aef8031d59fddfa4c2aab4f6ce75e8fc002f" gracePeriod=30 Jul 16 17:26:22.801947 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:22.801923 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:26:22.921146 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:22.921078 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b563d02-f2b6-4186-8c24-97584eced510-workload-socket\") pod \"4b563d02-f2b6-4186-8c24-97584eced510\" (UID: \"4b563d02-f2b6-4186-8c24-97584eced510\") " Jul 16 17:26:22.921146 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:22.921120 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sdnpg\" (UniqueName: \"kubernetes.io/projected/4b563d02-f2b6-4186-8c24-97584eced510-kube-api-access-sdnpg\") pod \"4b563d02-f2b6-4186-8c24-97584eced510\" (UID: \"4b563d02-f2b6-4186-8c24-97584eced510\") " Jul 16 17:26:22.921146 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:22.921139 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b563d02-f2b6-4186-8c24-97584eced510-istio-podinfo\") pod \"4b563d02-f2b6-4186-8c24-97584eced510\" (UID: \"4b563d02-f2b6-4186-8c24-97584eced510\") " Jul 16 17:26:22.921384 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:22.921153 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b563d02-f2b6-4186-8c24-97584eced510-credential-socket\") pod \"4b563d02-f2b6-4186-8c24-97584eced510\" (UID: \"4b563d02-f2b6-4186-8c24-97584eced510\") " Jul 16 17:26:22.921384 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:22.921193 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b563d02-f2b6-4186-8c24-97584eced510-istio-data\") pod \"4b563d02-f2b6-4186-8c24-97584eced510\" (UID: \"4b563d02-f2b6-4186-8c24-97584eced510\") " Jul 16 17:26:22.921384 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:22.921228 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b563d02-f2b6-4186-8c24-97584eced510-istio-envoy\") pod \"4b563d02-f2b6-4186-8c24-97584eced510\" (UID: \"4b563d02-f2b6-4186-8c24-97584eced510\") " Jul 16 17:26:22.921384 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:22.921242 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b563d02-f2b6-4186-8c24-97584eced510-istiod-ca-cert\") pod \"4b563d02-f2b6-4186-8c24-97584eced510\" (UID: \"4b563d02-f2b6-4186-8c24-97584eced510\") " Jul 16 17:26:22.921384 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:22.921289 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b563d02-f2b6-4186-8c24-97584eced510-workload-certs\") pod \"4b563d02-f2b6-4186-8c24-97584eced510\" (UID: \"4b563d02-f2b6-4186-8c24-97584eced510\") " Jul 16 17:26:22.921384 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:22.921328 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b563d02-f2b6-4186-8c24-97584eced510-istio-token\") pod \"4b563d02-f2b6-4186-8c24-97584eced510\" (UID: \"4b563d02-f2b6-4186-8c24-97584eced510\") " Jul 16 17:26:22.921690 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:22.921387 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b563d02-f2b6-4186-8c24-97584eced510-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4b563d02-f2b6-4186-8c24-97584eced510" (UID: "4b563d02-f2b6-4186-8c24-97584eced510"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:26:22.921690 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:22.921552 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b563d02-f2b6-4186-8c24-97584eced510-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4b563d02-f2b6-4186-8c24-97584eced510" (UID: "4b563d02-f2b6-4186-8c24-97584eced510"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:26:22.921690 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:22.921633 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b563d02-f2b6-4186-8c24-97584eced510-workload-socket\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:26:22.921690 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:22.921651 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b563d02-f2b6-4186-8c24-97584eced510-credential-socket\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:26:22.921690 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:22.921639 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b563d02-f2b6-4186-8c24-97584eced510-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4b563d02-f2b6-4186-8c24-97584eced510" (UID: "4b563d02-f2b6-4186-8c24-97584eced510"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:26:22.921985 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:22.921890 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b563d02-f2b6-4186-8c24-97584eced510-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4b563d02-f2b6-4186-8c24-97584eced510" (UID: "4b563d02-f2b6-4186-8c24-97584eced510"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:26:22.922190 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:22.922161 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b563d02-f2b6-4186-8c24-97584eced510-istio-data" (OuterVolumeSpecName: "istio-data") pod "4b563d02-f2b6-4186-8c24-97584eced510" (UID: "4b563d02-f2b6-4186-8c24-97584eced510"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:26:22.923471 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:22.923448 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b563d02-f2b6-4186-8c24-97584eced510-kube-api-access-sdnpg" (OuterVolumeSpecName: "kube-api-access-sdnpg") pod "4b563d02-f2b6-4186-8c24-97584eced510" (UID: "4b563d02-f2b6-4186-8c24-97584eced510"). InnerVolumeSpecName "kube-api-access-sdnpg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:26:22.923554 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:22.923448 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4b563d02-f2b6-4186-8c24-97584eced510-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4b563d02-f2b6-4186-8c24-97584eced510" (UID: "4b563d02-f2b6-4186-8c24-97584eced510"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 17:26:22.923554 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:22.923512 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b563d02-f2b6-4186-8c24-97584eced510-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4b563d02-f2b6-4186-8c24-97584eced510" (UID: "4b563d02-f2b6-4186-8c24-97584eced510"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:26:22.923701 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:22.923687 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b563d02-f2b6-4186-8c24-97584eced510-istio-token" (OuterVolumeSpecName: "istio-token") pod "4b563d02-f2b6-4186-8c24-97584eced510" (UID: "4b563d02-f2b6-4186-8c24-97584eced510"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:26:23.022339 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:23.022316 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b563d02-f2b6-4186-8c24-97584eced510-istio-token\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:26:23.022339 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:23.022335 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sdnpg\" (UniqueName: \"kubernetes.io/projected/4b563d02-f2b6-4186-8c24-97584eced510-kube-api-access-sdnpg\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:26:23.022538 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:23.022345 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b563d02-f2b6-4186-8c24-97584eced510-istio-podinfo\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:26:23.022538 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:23.022353 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b563d02-f2b6-4186-8c24-97584eced510-istio-data\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:26:23.022538 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:23.022361 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b563d02-f2b6-4186-8c24-97584eced510-istio-envoy\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:26:23.022538 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:23.022369 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b563d02-f2b6-4186-8c24-97584eced510-istiod-ca-cert\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:26:23.022538 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:23.022377 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b563d02-f2b6-4186-8c24-97584eced510-workload-certs\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:26:23.636067 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:23.636031 2571 generic.go:356] "Generic (PLEG): container finished" podID="4b563d02-f2b6-4186-8c24-97584eced510" containerID="55b1f2454b8ba8e621074d48c342aef8031d59fddfa4c2aab4f6ce75e8fc002f" exitCode=0 Jul 16 17:26:23.636265 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:23.636078 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" Jul 16 17:26:23.636265 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:23.636095 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" event={"ID":"4b563d02-f2b6-4186-8c24-97584eced510","Type":"ContainerDied","Data":"55b1f2454b8ba8e621074d48c342aef8031d59fddfa4c2aab4f6ce75e8fc002f"} Jul 16 17:26:23.636265 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:23.636146 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl" event={"ID":"4b563d02-f2b6-4186-8c24-97584eced510","Type":"ContainerDied","Data":"cb90e931513044cb711bfe51a4d9febe5b73159191f81f77c7bb2fc71c22e357"} Jul 16 17:26:23.636265 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:23.636173 2571 scope.go:117] "RemoveContainer" containerID="55b1f2454b8ba8e621074d48c342aef8031d59fddfa4c2aab4f6ce75e8fc002f" Jul 16 17:26:23.645395 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:23.645380 2571 scope.go:117] "RemoveContainer" containerID="e8b52235a588dd0d117da16aec16709662c96492bb84d940401eb32a7bdb0270" Jul 16 17:26:23.653266 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:23.653250 2571 scope.go:117] "RemoveContainer" containerID="55b1f2454b8ba8e621074d48c342aef8031d59fddfa4c2aab4f6ce75e8fc002f" Jul 16 17:26:23.653503 ip-10-0-128-50 kubenswrapper[2571]: E0716 17:26:23.653483 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55b1f2454b8ba8e621074d48c342aef8031d59fddfa4c2aab4f6ce75e8fc002f\": container with ID starting with 55b1f2454b8ba8e621074d48c342aef8031d59fddfa4c2aab4f6ce75e8fc002f not found: ID does not exist" containerID="55b1f2454b8ba8e621074d48c342aef8031d59fddfa4c2aab4f6ce75e8fc002f" Jul 16 17:26:23.653597 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:23.653509 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55b1f2454b8ba8e621074d48c342aef8031d59fddfa4c2aab4f6ce75e8fc002f"} err="failed to get container status \"55b1f2454b8ba8e621074d48c342aef8031d59fddfa4c2aab4f6ce75e8fc002f\": rpc error: code = NotFound desc = could not find container \"55b1f2454b8ba8e621074d48c342aef8031d59fddfa4c2aab4f6ce75e8fc002f\": container with ID starting with 55b1f2454b8ba8e621074d48c342aef8031d59fddfa4c2aab4f6ce75e8fc002f not found: ID does not exist" Jul 16 17:26:23.653597 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:23.653525 2571 scope.go:117] "RemoveContainer" containerID="e8b52235a588dd0d117da16aec16709662c96492bb84d940401eb32a7bdb0270" Jul 16 17:26:23.653778 ip-10-0-128-50 kubenswrapper[2571]: E0716 17:26:23.653716 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8b52235a588dd0d117da16aec16709662c96492bb84d940401eb32a7bdb0270\": container with ID starting with e8b52235a588dd0d117da16aec16709662c96492bb84d940401eb32a7bdb0270 not found: ID does not exist" containerID="e8b52235a588dd0d117da16aec16709662c96492bb84d940401eb32a7bdb0270" Jul 16 17:26:23.653851 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:23.653788 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8b52235a588dd0d117da16aec16709662c96492bb84d940401eb32a7bdb0270"} err="failed to get container status \"e8b52235a588dd0d117da16aec16709662c96492bb84d940401eb32a7bdb0270\": rpc error: code = NotFound desc = could not find container \"e8b52235a588dd0d117da16aec16709662c96492bb84d940401eb32a7bdb0270\": container with ID starting with e8b52235a588dd0d117da16aec16709662c96492bb84d940401eb32a7bdb0270 not found: ID does not exist" Jul 16 17:26:23.671025 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:23.671003 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl"] Jul 16 17:26:23.684694 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:23.684673 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-36dc65-openshift-default-d789bcc89-tnlcl"] Jul 16 17:26:24.777408 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:24.777377 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b563d02-f2b6-4186-8c24-97584eced510" path="/var/lib/kubelet/pods/4b563d02-f2b6-4186-8c24-97584eced510/volumes" Jul 16 17:26:35.972395 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:35.972341 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7"] Jul 16 17:26:35.972869 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:35.972703 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b563d02-f2b6-4186-8c24-97584eced510" containerName="istio-proxy" Jul 16 17:26:35.972869 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:35.972714 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b563d02-f2b6-4186-8c24-97584eced510" containerName="istio-proxy" Jul 16 17:26:35.972869 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:35.972721 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b563d02-f2b6-4186-8c24-97584eced510" containerName="istio-proxy" Jul 16 17:26:35.972869 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:35.972741 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b563d02-f2b6-4186-8c24-97584eced510" containerName="istio-proxy" Jul 16 17:26:35.972869 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:35.972798 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b563d02-f2b6-4186-8c24-97584eced510" containerName="istio-proxy" Jul 16 17:26:35.972869 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:35.972807 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b563d02-f2b6-4186-8c24-97584eced510" containerName="istio-proxy" Jul 16 17:26:35.975990 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:35.975972 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" Jul 16 17:26:35.980548 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:35.980528 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 16 17:26:35.980658 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:35.980562 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-b44c0f-openshift-default-dockercfg-w5lnq\"" Jul 16 17:26:36.004007 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:36.003981 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7"] Jul 16 17:26:36.020184 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:36.020159 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82e15968-cfde-4887-97c5-33719f95bf87-workload-certs\") pod \"e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7\" (UID: \"82e15968-cfde-4887-97c5-33719f95bf87\") " pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" Jul 16 17:26:36.020287 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:36.020185 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82e15968-cfde-4887-97c5-33719f95bf87-istio-data\") pod \"e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7\" (UID: \"82e15968-cfde-4887-97c5-33719f95bf87\") " pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" Jul 16 17:26:36.020287 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:36.020202 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82e15968-cfde-4887-97c5-33719f95bf87-istio-token\") pod \"e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7\" (UID: \"82e15968-cfde-4887-97c5-33719f95bf87\") " pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" Jul 16 17:26:36.020287 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:36.020223 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbf9s\" (UniqueName: \"kubernetes.io/projected/82e15968-cfde-4887-97c5-33719f95bf87-kube-api-access-xbf9s\") pod \"e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7\" (UID: \"82e15968-cfde-4887-97c5-33719f95bf87\") " pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" Jul 16 17:26:36.020414 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:36.020295 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82e15968-cfde-4887-97c5-33719f95bf87-istio-podinfo\") pod \"e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7\" (UID: \"82e15968-cfde-4887-97c5-33719f95bf87\") " pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" Jul 16 17:26:36.020414 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:36.020370 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82e15968-cfde-4887-97c5-33719f95bf87-credential-socket\") pod \"e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7\" (UID: \"82e15968-cfde-4887-97c5-33719f95bf87\") " pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" Jul 16 17:26:36.020414 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:36.020397 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82e15968-cfde-4887-97c5-33719f95bf87-istio-envoy\") pod \"e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7\" (UID: \"82e15968-cfde-4887-97c5-33719f95bf87\") " pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" Jul 16 17:26:36.020527 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:36.020435 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82e15968-cfde-4887-97c5-33719f95bf87-workload-socket\") pod \"e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7\" (UID: \"82e15968-cfde-4887-97c5-33719f95bf87\") " pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" Jul 16 17:26:36.020527 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:36.020453 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82e15968-cfde-4887-97c5-33719f95bf87-istiod-ca-cert\") pod \"e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7\" (UID: \"82e15968-cfde-4887-97c5-33719f95bf87\") " pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" Jul 16 17:26:36.121612 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:36.121588 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82e15968-cfde-4887-97c5-33719f95bf87-istio-podinfo\") pod \"e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7\" (UID: \"82e15968-cfde-4887-97c5-33719f95bf87\") " pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" Jul 16 17:26:36.121780 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:36.121630 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82e15968-cfde-4887-97c5-33719f95bf87-credential-socket\") pod \"e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7\" (UID: \"82e15968-cfde-4887-97c5-33719f95bf87\") " pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" Jul 16 17:26:36.121780 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:36.121648 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82e15968-cfde-4887-97c5-33719f95bf87-istio-envoy\") pod \"e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7\" (UID: \"82e15968-cfde-4887-97c5-33719f95bf87\") " pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" Jul 16 17:26:36.121780 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:36.121690 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82e15968-cfde-4887-97c5-33719f95bf87-workload-socket\") pod \"e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7\" (UID: \"82e15968-cfde-4887-97c5-33719f95bf87\") " pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" Jul 16 17:26:36.121780 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:36.121716 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82e15968-cfde-4887-97c5-33719f95bf87-istiod-ca-cert\") pod \"e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7\" (UID: \"82e15968-cfde-4887-97c5-33719f95bf87\") " pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" Jul 16 17:26:36.122000 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:36.121890 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82e15968-cfde-4887-97c5-33719f95bf87-workload-certs\") pod \"e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7\" (UID: \"82e15968-cfde-4887-97c5-33719f95bf87\") " pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" Jul 16 17:26:36.122000 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:36.121922 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82e15968-cfde-4887-97c5-33719f95bf87-istio-data\") pod \"e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7\" (UID: \"82e15968-cfde-4887-97c5-33719f95bf87\") " pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" Jul 16 17:26:36.122000 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:36.121948 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82e15968-cfde-4887-97c5-33719f95bf87-istio-token\") pod \"e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7\" (UID: \"82e15968-cfde-4887-97c5-33719f95bf87\") " pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" Jul 16 17:26:36.122000 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:36.121984 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbf9s\" (UniqueName: \"kubernetes.io/projected/82e15968-cfde-4887-97c5-33719f95bf87-kube-api-access-xbf9s\") pod \"e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7\" (UID: \"82e15968-cfde-4887-97c5-33719f95bf87\") " pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" Jul 16 17:26:36.122203 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:36.122052 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82e15968-cfde-4887-97c5-33719f95bf87-credential-socket\") pod \"e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7\" (UID: \"82e15968-cfde-4887-97c5-33719f95bf87\") " pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" Jul 16 17:26:36.122203 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:36.122122 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82e15968-cfde-4887-97c5-33719f95bf87-workload-socket\") pod \"e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7\" (UID: \"82e15968-cfde-4887-97c5-33719f95bf87\") " pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" Jul 16 17:26:36.122203 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:36.122193 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82e15968-cfde-4887-97c5-33719f95bf87-istio-data\") pod \"e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7\" (UID: \"82e15968-cfde-4887-97c5-33719f95bf87\") " pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" Jul 16 17:26:36.122353 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:36.122246 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82e15968-cfde-4887-97c5-33719f95bf87-workload-certs\") pod \"e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7\" (UID: \"82e15968-cfde-4887-97c5-33719f95bf87\") " pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" Jul 16 17:26:36.122645 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:36.122626 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82e15968-cfde-4887-97c5-33719f95bf87-istiod-ca-cert\") pod \"e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7\" (UID: \"82e15968-cfde-4887-97c5-33719f95bf87\") " pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" Jul 16 17:26:36.123882 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:36.123863 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82e15968-cfde-4887-97c5-33719f95bf87-istio-envoy\") pod \"e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7\" (UID: \"82e15968-cfde-4887-97c5-33719f95bf87\") " pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" Jul 16 17:26:36.124190 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:36.124170 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82e15968-cfde-4887-97c5-33719f95bf87-istio-podinfo\") pod \"e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7\" (UID: \"82e15968-cfde-4887-97c5-33719f95bf87\") " pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" Jul 16 17:26:36.140449 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:36.140422 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbf9s\" (UniqueName: \"kubernetes.io/projected/82e15968-cfde-4887-97c5-33719f95bf87-kube-api-access-xbf9s\") pod \"e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7\" (UID: \"82e15968-cfde-4887-97c5-33719f95bf87\") " pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" Jul 16 17:26:36.144020 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:36.144001 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82e15968-cfde-4887-97c5-33719f95bf87-istio-token\") pod \"e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7\" (UID: \"82e15968-cfde-4887-97c5-33719f95bf87\") " pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" Jul 16 17:26:36.285817 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:36.285753 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" Jul 16 17:26:36.460658 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:36.460630 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7"] Jul 16 17:26:36.463766 ip-10-0-128-50 kubenswrapper[2571]: W0716 17:26:36.463736 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82e15968_cfde_4887_97c5_33719f95bf87.slice/crio-6bb9c3becded1afa08e5f9c637ac57abb328fcf1e55c58556d4da3b2e9503faf WatchSource:0}: Error finding container 6bb9c3becded1afa08e5f9c637ac57abb328fcf1e55c58556d4da3b2e9503faf: Status 404 returned error can't find the container with id 6bb9c3becded1afa08e5f9c637ac57abb328fcf1e55c58556d4da3b2e9503faf Jul 16 17:26:36.465759 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:36.465711 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 17:26:36.465843 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:36.465788 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 17:26:36.465843 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:36.465819 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 17:26:36.707773 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:36.707719 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" event={"ID":"82e15968-cfde-4887-97c5-33719f95bf87","Type":"ContainerStarted","Data":"78816b218a2e5de77733f805be121be06f0cfb658ce55934081e3a848c206463"} Jul 16 17:26:36.707924 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:36.707807 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" event={"ID":"82e15968-cfde-4887-97c5-33719f95bf87","Type":"ContainerStarted","Data":"6bb9c3becded1afa08e5f9c637ac57abb328fcf1e55c58556d4da3b2e9503faf"} Jul 16 17:26:36.774701 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:36.774657 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" podStartSLOduration=1.77464252 podStartE2EDuration="1.77464252s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:26:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:26:36.771079113 +0000 UTC m=+2148.512654410" watchObservedRunningTime="2026-07-16 17:26:36.77464252 +0000 UTC m=+2148.516217814" Jul 16 17:26:37.286873 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:37.286845 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" Jul 16 17:26:37.292487 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:37.292463 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" Jul 16 17:26:37.712687 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:37.712656 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" Jul 16 17:26:37.713639 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:37.713623 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" Jul 16 17:26:52.714463 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:52.714427 2571 patch_prober.go:28] interesting pod/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 16 17:26:52.714881 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:52.714486 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" podUID="82e15968-cfde-4887-97c5-33719f95bf87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 16 17:26:52.793135 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:52.793110 2571 generic.go:356] "Generic (PLEG): container finished" podID="82e15968-cfde-4887-97c5-33719f95bf87" containerID="78816b218a2e5de77733f805be121be06f0cfb658ce55934081e3a848c206463" exitCode=0 Jul 16 17:26:52.793306 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:52.793180 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" event={"ID":"82e15968-cfde-4887-97c5-33719f95bf87","Type":"ContainerDied","Data":"78816b218a2e5de77733f805be121be06f0cfb658ce55934081e3a848c206463"} Jul 16 17:26:52.793548 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:52.793531 2571 scope.go:117] "RemoveContainer" containerID="78816b218a2e5de77733f805be121be06f0cfb658ce55934081e3a848c206463" Jul 16 17:26:52.794549 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:52.794516 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 17:26:52.794620 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:52.794602 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 17:26:52.794670 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:52.794656 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 17:26:53.803347 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:53.803313 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" event={"ID":"82e15968-cfde-4887-97c5-33719f95bf87","Type":"ContainerStarted","Data":"082f3293d925dbfc517408647b98ab19cbae24c19a030a8814faf316b220766d"} Jul 16 17:26:54.286095 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:54.286063 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" Jul 16 17:26:54.286264 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:54.286207 2571 patch_prober.go:28] interesting pod/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 16 17:26:54.286264 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:54.286250 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" podUID="82e15968-cfde-4887-97c5-33719f95bf87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 16 17:26:55.286031 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:55.285998 2571 patch_prober.go:28] interesting pod/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 16 17:26:55.286392 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:55.286046 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" podUID="82e15968-cfde-4887-97c5-33719f95bf87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 16 17:26:56.286296 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:56.286256 2571 patch_prober.go:28] interesting pod/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 16 17:26:56.286773 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:56.286327 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" podUID="82e15968-cfde-4887-97c5-33719f95bf87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 16 17:26:57.286619 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:57.286582 2571 patch_prober.go:28] interesting pod/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 16 17:26:57.287074 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:57.286638 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" podUID="82e15968-cfde-4887-97c5-33719f95bf87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 16 17:26:58.290080 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:58.290052 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" Jul 16 17:26:58.290471 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:58.290282 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" Jul 16 17:26:58.291017 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:26:58.290993 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" Jul 16 17:27:46.601134 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:27:46.601102 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-8d3a22/auth-test-model-8d3a22-kserve-78cc696f99-p4ppr"] Jul 16 17:27:46.604715 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:27:46.604693 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8d3a22/auth-test-model-8d3a22-kserve-78cc696f99-p4ppr" Jul 16 17:27:46.612540 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:27:46.612518 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8d3a22\"/\"kube-root-ca.crt\"" Jul 16 17:27:46.613621 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:27:46.613601 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8d3a22\"/\"openshift-service-ca.crt\"" Jul 16 17:27:46.613766 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:27:46.613601 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8d3a22\"/\"auth-test-model-8d3a22-kserve-self-signed-certs\"" Jul 16 17:27:46.613863 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:27:46.613848 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8d3a22\"/\"default-dockercfg-4xt5g\"" Jul 16 17:27:46.620008 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:27:46.619981 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8d3a22/auth-test-model-8d3a22-kserve-78cc696f99-p4ppr"] Jul 16 17:27:46.769649 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:27:46.769614 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrxxz\" (UniqueName: \"kubernetes.io/projected/2dd6fd20-20ad-4d6d-9970-a2881ed5e74d-kube-api-access-wrxxz\") pod \"auth-test-model-8d3a22-kserve-78cc696f99-p4ppr\" (UID: \"2dd6fd20-20ad-4d6d-9970-a2881ed5e74d\") " pod="ai-tenant-e2e-shared-b-8d3a22/auth-test-model-8d3a22-kserve-78cc696f99-p4ppr" Jul 16 17:27:46.769792 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:27:46.769657 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2dd6fd20-20ad-4d6d-9970-a2881ed5e74d-tls-certs\") pod \"auth-test-model-8d3a22-kserve-78cc696f99-p4ppr\" (UID: \"2dd6fd20-20ad-4d6d-9970-a2881ed5e74d\") " pod="ai-tenant-e2e-shared-b-8d3a22/auth-test-model-8d3a22-kserve-78cc696f99-p4ppr" Jul 16 17:27:46.769792 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:27:46.769674 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2dd6fd20-20ad-4d6d-9970-a2881ed5e74d-home\") pod \"auth-test-model-8d3a22-kserve-78cc696f99-p4ppr\" (UID: \"2dd6fd20-20ad-4d6d-9970-a2881ed5e74d\") " pod="ai-tenant-e2e-shared-b-8d3a22/auth-test-model-8d3a22-kserve-78cc696f99-p4ppr" Jul 16 17:27:46.769792 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:27:46.769766 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2dd6fd20-20ad-4d6d-9970-a2881ed5e74d-dshm\") pod \"auth-test-model-8d3a22-kserve-78cc696f99-p4ppr\" (UID: \"2dd6fd20-20ad-4d6d-9970-a2881ed5e74d\") " pod="ai-tenant-e2e-shared-b-8d3a22/auth-test-model-8d3a22-kserve-78cc696f99-p4ppr" Jul 16 17:27:46.769792 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:27:46.769786 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2dd6fd20-20ad-4d6d-9970-a2881ed5e74d-model-cache\") pod \"auth-test-model-8d3a22-kserve-78cc696f99-p4ppr\" (UID: \"2dd6fd20-20ad-4d6d-9970-a2881ed5e74d\") " pod="ai-tenant-e2e-shared-b-8d3a22/auth-test-model-8d3a22-kserve-78cc696f99-p4ppr" Jul 16 17:27:46.769961 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:27:46.769824 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2dd6fd20-20ad-4d6d-9970-a2881ed5e74d-tmp-dir\") pod \"auth-test-model-8d3a22-kserve-78cc696f99-p4ppr\" (UID: \"2dd6fd20-20ad-4d6d-9970-a2881ed5e74d\") " pod="ai-tenant-e2e-shared-b-8d3a22/auth-test-model-8d3a22-kserve-78cc696f99-p4ppr" Jul 16 17:27:46.870750 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:27:46.870650 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrxxz\" (UniqueName: \"kubernetes.io/projected/2dd6fd20-20ad-4d6d-9970-a2881ed5e74d-kube-api-access-wrxxz\") pod \"auth-test-model-8d3a22-kserve-78cc696f99-p4ppr\" (UID: \"2dd6fd20-20ad-4d6d-9970-a2881ed5e74d\") " pod="ai-tenant-e2e-shared-b-8d3a22/auth-test-model-8d3a22-kserve-78cc696f99-p4ppr" Jul 16 17:27:46.870750 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:27:46.870701 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2dd6fd20-20ad-4d6d-9970-a2881ed5e74d-tls-certs\") pod \"auth-test-model-8d3a22-kserve-78cc696f99-p4ppr\" (UID: \"2dd6fd20-20ad-4d6d-9970-a2881ed5e74d\") " pod="ai-tenant-e2e-shared-b-8d3a22/auth-test-model-8d3a22-kserve-78cc696f99-p4ppr" Jul 16 17:27:46.870932 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:27:46.870753 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2dd6fd20-20ad-4d6d-9970-a2881ed5e74d-home\") pod \"auth-test-model-8d3a22-kserve-78cc696f99-p4ppr\" (UID: \"2dd6fd20-20ad-4d6d-9970-a2881ed5e74d\") " pod="ai-tenant-e2e-shared-b-8d3a22/auth-test-model-8d3a22-kserve-78cc696f99-p4ppr" Jul 16 17:27:46.870932 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:27:46.870793 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2dd6fd20-20ad-4d6d-9970-a2881ed5e74d-dshm\") pod \"auth-test-model-8d3a22-kserve-78cc696f99-p4ppr\" (UID: \"2dd6fd20-20ad-4d6d-9970-a2881ed5e74d\") " pod="ai-tenant-e2e-shared-b-8d3a22/auth-test-model-8d3a22-kserve-78cc696f99-p4ppr" Jul 16 17:27:46.870932 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:27:46.870817 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2dd6fd20-20ad-4d6d-9970-a2881ed5e74d-model-cache\") pod \"auth-test-model-8d3a22-kserve-78cc696f99-p4ppr\" (UID: \"2dd6fd20-20ad-4d6d-9970-a2881ed5e74d\") " pod="ai-tenant-e2e-shared-b-8d3a22/auth-test-model-8d3a22-kserve-78cc696f99-p4ppr" Jul 16 17:27:46.871084 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:27:46.870932 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2dd6fd20-20ad-4d6d-9970-a2881ed5e74d-tmp-dir\") pod \"auth-test-model-8d3a22-kserve-78cc696f99-p4ppr\" (UID: \"2dd6fd20-20ad-4d6d-9970-a2881ed5e74d\") " pod="ai-tenant-e2e-shared-b-8d3a22/auth-test-model-8d3a22-kserve-78cc696f99-p4ppr" Jul 16 17:27:46.871207 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:27:46.871186 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2dd6fd20-20ad-4d6d-9970-a2881ed5e74d-home\") pod \"auth-test-model-8d3a22-kserve-78cc696f99-p4ppr\" (UID: \"2dd6fd20-20ad-4d6d-9970-a2881ed5e74d\") " pod="ai-tenant-e2e-shared-b-8d3a22/auth-test-model-8d3a22-kserve-78cc696f99-p4ppr" Jul 16 17:27:46.871297 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:27:46.871259 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2dd6fd20-20ad-4d6d-9970-a2881ed5e74d-model-cache\") pod \"auth-test-model-8d3a22-kserve-78cc696f99-p4ppr\" (UID: \"2dd6fd20-20ad-4d6d-9970-a2881ed5e74d\") " pod="ai-tenant-e2e-shared-b-8d3a22/auth-test-model-8d3a22-kserve-78cc696f99-p4ppr" Jul 16 17:27:46.871297 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:27:46.871283 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2dd6fd20-20ad-4d6d-9970-a2881ed5e74d-tmp-dir\") pod \"auth-test-model-8d3a22-kserve-78cc696f99-p4ppr\" (UID: \"2dd6fd20-20ad-4d6d-9970-a2881ed5e74d\") " pod="ai-tenant-e2e-shared-b-8d3a22/auth-test-model-8d3a22-kserve-78cc696f99-p4ppr" Jul 16 17:27:46.872967 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:27:46.872947 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2dd6fd20-20ad-4d6d-9970-a2881ed5e74d-dshm\") pod \"auth-test-model-8d3a22-kserve-78cc696f99-p4ppr\" (UID: \"2dd6fd20-20ad-4d6d-9970-a2881ed5e74d\") " pod="ai-tenant-e2e-shared-b-8d3a22/auth-test-model-8d3a22-kserve-78cc696f99-p4ppr" Jul 16 17:27:46.873216 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:27:46.873201 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2dd6fd20-20ad-4d6d-9970-a2881ed5e74d-tls-certs\") pod \"auth-test-model-8d3a22-kserve-78cc696f99-p4ppr\" (UID: \"2dd6fd20-20ad-4d6d-9970-a2881ed5e74d\") " pod="ai-tenant-e2e-shared-b-8d3a22/auth-test-model-8d3a22-kserve-78cc696f99-p4ppr" Jul 16 17:27:46.881624 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:27:46.881603 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrxxz\" (UniqueName: \"kubernetes.io/projected/2dd6fd20-20ad-4d6d-9970-a2881ed5e74d-kube-api-access-wrxxz\") pod \"auth-test-model-8d3a22-kserve-78cc696f99-p4ppr\" (UID: \"2dd6fd20-20ad-4d6d-9970-a2881ed5e74d\") " pod="ai-tenant-e2e-shared-b-8d3a22/auth-test-model-8d3a22-kserve-78cc696f99-p4ppr" Jul 16 17:27:46.913489 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:27:46.913461 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8d3a22/auth-test-model-8d3a22-kserve-78cc696f99-p4ppr" Jul 16 17:27:47.068013 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:27:47.067987 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8d3a22/auth-test-model-8d3a22-kserve-78cc696f99-p4ppr"] Jul 16 17:27:47.069410 ip-10-0-128-50 kubenswrapper[2571]: W0716 17:27:47.069384 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2dd6fd20_20ad_4d6d_9970_a2881ed5e74d.slice/crio-1773db40e6cc90dfae70e230d190e49504e13285ac4bd22fba488e58c6078cfc WatchSource:0}: Error finding container 1773db40e6cc90dfae70e230d190e49504e13285ac4bd22fba488e58c6078cfc: Status 404 returned error can't find the container with id 1773db40e6cc90dfae70e230d190e49504e13285ac4bd22fba488e58c6078cfc Jul 16 17:27:48.081116 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:27:48.081074 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8d3a22/auth-test-model-8d3a22-kserve-78cc696f99-p4ppr" event={"ID":"2dd6fd20-20ad-4d6d-9970-a2881ed5e74d","Type":"ContainerStarted","Data":"6857ba91ea2b66da500f87c2e0241237f014dcf1f0cab03b1fda592f14a2758e"} Jul 16 17:27:48.081116 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:27:48.081121 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8d3a22/auth-test-model-8d3a22-kserve-78cc696f99-p4ppr" event={"ID":"2dd6fd20-20ad-4d6d-9970-a2881ed5e74d","Type":"ContainerStarted","Data":"1773db40e6cc90dfae70e230d190e49504e13285ac4bd22fba488e58c6078cfc"} Jul 16 17:27:48.106905 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:27:48.106855 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-8d3a22/auth-test-model-8d3a22-kserve-78cc696f99-p4ppr" podStartSLOduration=2.106841306 podStartE2EDuration="2.106841306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:27:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:27:48.104363867 +0000 UTC m=+2219.845939166" watchObservedRunningTime="2026-07-16 17:27:48.106841306 +0000 UTC m=+2219.848416603" Jul 16 17:27:56.914345 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:27:56.914312 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-8d3a22/auth-test-model-8d3a22-kserve-78cc696f99-p4ppr" Jul 16 17:27:56.914345 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:27:56.914353 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-8d3a22/auth-test-model-8d3a22-kserve-78cc696f99-p4ppr" Jul 16 17:27:56.926112 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:27:56.926088 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-8d3a22/auth-test-model-8d3a22-kserve-78cc696f99-p4ppr" Jul 16 17:27:57.139198 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:27:57.139172 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-8d3a22/auth-test-model-8d3a22-kserve-78cc696f99-p4ppr" Jul 16 17:29:58.730080 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:29:58.730002 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8d3a22/auth-test-model-8d3a22-kserve-78cc696f99-p4ppr"] Jul 16 17:30:13.749235 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:30:13.749168 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-8d3a22/auth-test-model-8d3a22-kserve-78cc696f99-p4ppr" podUID="2dd6fd20-20ad-4d6d-9970-a2881ed5e74d" containerName="main" containerID="cri-o://6857ba91ea2b66da500f87c2e0241237f014dcf1f0cab03b1fda592f14a2758e" gracePeriod=45 Jul 16 17:30:13.999950 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:30:13.999889 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8d3a22/auth-test-model-8d3a22-kserve-78cc696f99-p4ppr" Jul 16 17:30:14.097399 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:30:14.097369 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrxxz\" (UniqueName: \"kubernetes.io/projected/2dd6fd20-20ad-4d6d-9970-a2881ed5e74d-kube-api-access-wrxxz\") pod \"2dd6fd20-20ad-4d6d-9970-a2881ed5e74d\" (UID: \"2dd6fd20-20ad-4d6d-9970-a2881ed5e74d\") " Jul 16 17:30:14.097557 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:30:14.097415 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2dd6fd20-20ad-4d6d-9970-a2881ed5e74d-dshm\") pod \"2dd6fd20-20ad-4d6d-9970-a2881ed5e74d\" (UID: \"2dd6fd20-20ad-4d6d-9970-a2881ed5e74d\") " Jul 16 17:30:14.097557 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:30:14.097461 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2dd6fd20-20ad-4d6d-9970-a2881ed5e74d-tls-certs\") pod \"2dd6fd20-20ad-4d6d-9970-a2881ed5e74d\" (UID: \"2dd6fd20-20ad-4d6d-9970-a2881ed5e74d\") " Jul 16 17:30:14.097557 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:30:14.097494 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2dd6fd20-20ad-4d6d-9970-a2881ed5e74d-home\") pod \"2dd6fd20-20ad-4d6d-9970-a2881ed5e74d\" (UID: \"2dd6fd20-20ad-4d6d-9970-a2881ed5e74d\") " Jul 16 17:30:14.097557 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:30:14.097542 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2dd6fd20-20ad-4d6d-9970-a2881ed5e74d-model-cache\") pod \"2dd6fd20-20ad-4d6d-9970-a2881ed5e74d\" (UID: \"2dd6fd20-20ad-4d6d-9970-a2881ed5e74d\") " Jul 16 17:30:14.097787 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:30:14.097565 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2dd6fd20-20ad-4d6d-9970-a2881ed5e74d-tmp-dir\") pod \"2dd6fd20-20ad-4d6d-9970-a2881ed5e74d\" (UID: \"2dd6fd20-20ad-4d6d-9970-a2881ed5e74d\") " Jul 16 17:30:14.097787 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:30:14.097760 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2dd6fd20-20ad-4d6d-9970-a2881ed5e74d-home" (OuterVolumeSpecName: "home") pod "2dd6fd20-20ad-4d6d-9970-a2881ed5e74d" (UID: "2dd6fd20-20ad-4d6d-9970-a2881ed5e74d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:30:14.097888 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:30:14.097854 2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2dd6fd20-20ad-4d6d-9970-a2881ed5e74d-home\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:30:14.097948 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:30:14.097929 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2dd6fd20-20ad-4d6d-9970-a2881ed5e74d-model-cache" (OuterVolumeSpecName: "model-cache") pod "2dd6fd20-20ad-4d6d-9970-a2881ed5e74d" (UID: "2dd6fd20-20ad-4d6d-9970-a2881ed5e74d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:30:14.098127 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:30:14.098046 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2dd6fd20-20ad-4d6d-9970-a2881ed5e74d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2dd6fd20-20ad-4d6d-9970-a2881ed5e74d" (UID: "2dd6fd20-20ad-4d6d-9970-a2881ed5e74d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:30:14.100142 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:30:14.100112 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2dd6fd20-20ad-4d6d-9970-a2881ed5e74d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2dd6fd20-20ad-4d6d-9970-a2881ed5e74d" (UID: "2dd6fd20-20ad-4d6d-9970-a2881ed5e74d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:30:14.100142 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:30:14.100127 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2dd6fd20-20ad-4d6d-9970-a2881ed5e74d-kube-api-access-wrxxz" (OuterVolumeSpecName: "kube-api-access-wrxxz") pod "2dd6fd20-20ad-4d6d-9970-a2881ed5e74d" (UID: "2dd6fd20-20ad-4d6d-9970-a2881ed5e74d"). InnerVolumeSpecName "kube-api-access-wrxxz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:30:14.100273 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:30:14.100150 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2dd6fd20-20ad-4d6d-9970-a2881ed5e74d-dshm" (OuterVolumeSpecName: "dshm") pod "2dd6fd20-20ad-4d6d-9970-a2881ed5e74d" (UID: "2dd6fd20-20ad-4d6d-9970-a2881ed5e74d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:30:14.198366 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:30:14.198327 2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2dd6fd20-20ad-4d6d-9970-a2881ed5e74d-model-cache\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:30:14.198366 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:30:14.198361 2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2dd6fd20-20ad-4d6d-9970-a2881ed5e74d-tmp-dir\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:30:14.198366 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:30:14.198370 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrxxz\" (UniqueName: \"kubernetes.io/projected/2dd6fd20-20ad-4d6d-9970-a2881ed5e74d-kube-api-access-wrxxz\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:30:14.198366 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:30:14.198379 2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2dd6fd20-20ad-4d6d-9970-a2881ed5e74d-dshm\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:30:14.198646 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:30:14.198387 2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2dd6fd20-20ad-4d6d-9970-a2881ed5e74d-tls-certs\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:30:14.845465 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:30:14.845437 2571 generic.go:356] "Generic (PLEG): container finished" podID="2dd6fd20-20ad-4d6d-9970-a2881ed5e74d" containerID="6857ba91ea2b66da500f87c2e0241237f014dcf1f0cab03b1fda592f14a2758e" exitCode=0 Jul 16 17:30:14.845904 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:30:14.845480 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8d3a22/auth-test-model-8d3a22-kserve-78cc696f99-p4ppr" Jul 16 17:30:14.845904 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:30:14.845522 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8d3a22/auth-test-model-8d3a22-kserve-78cc696f99-p4ppr" event={"ID":"2dd6fd20-20ad-4d6d-9970-a2881ed5e74d","Type":"ContainerDied","Data":"6857ba91ea2b66da500f87c2e0241237f014dcf1f0cab03b1fda592f14a2758e"} Jul 16 17:30:14.845904 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:30:14.845562 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8d3a22/auth-test-model-8d3a22-kserve-78cc696f99-p4ppr" event={"ID":"2dd6fd20-20ad-4d6d-9970-a2881ed5e74d","Type":"ContainerDied","Data":"1773db40e6cc90dfae70e230d190e49504e13285ac4bd22fba488e58c6078cfc"} Jul 16 17:30:14.845904 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:30:14.845579 2571 scope.go:117] "RemoveContainer" containerID="6857ba91ea2b66da500f87c2e0241237f014dcf1f0cab03b1fda592f14a2758e" Jul 16 17:30:14.854681 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:30:14.854648 2571 scope.go:117] "RemoveContainer" containerID="6857ba91ea2b66da500f87c2e0241237f014dcf1f0cab03b1fda592f14a2758e" Jul 16 17:30:14.854928 ip-10-0-128-50 kubenswrapper[2571]: E0716 17:30:14.854912 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6857ba91ea2b66da500f87c2e0241237f014dcf1f0cab03b1fda592f14a2758e\": container with ID starting with 6857ba91ea2b66da500f87c2e0241237f014dcf1f0cab03b1fda592f14a2758e not found: ID does not exist" containerID="6857ba91ea2b66da500f87c2e0241237f014dcf1f0cab03b1fda592f14a2758e" Jul 16 17:30:14.854974 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:30:14.854936 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6857ba91ea2b66da500f87c2e0241237f014dcf1f0cab03b1fda592f14a2758e"} err="failed to get container status \"6857ba91ea2b66da500f87c2e0241237f014dcf1f0cab03b1fda592f14a2758e\": rpc error: code = NotFound desc = could not find container \"6857ba91ea2b66da500f87c2e0241237f014dcf1f0cab03b1fda592f14a2758e\": container with ID starting with 6857ba91ea2b66da500f87c2e0241237f014dcf1f0cab03b1fda592f14a2758e not found: ID does not exist" Jul 16 17:30:14.875034 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:30:14.875011 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8d3a22/auth-test-model-8d3a22-kserve-78cc696f99-p4ppr"] Jul 16 17:30:14.879925 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:30:14.879907 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-8d3a22/auth-test-model-8d3a22-kserve-78cc696f99-p4ppr"] Jul 16 17:30:16.777489 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:30:16.777460 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2dd6fd20-20ad-4d6d-9970-a2881ed5e74d" path="/var/lib/kubelet/pods/2dd6fd20-20ad-4d6d-9970-a2881ed5e74d/volumes" Jul 16 17:30:48.937581 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:30:48.937553 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r8cn4_f637d961-672d-4aa2-a49a-fd021c2d4df2/console-operator/2.log" Jul 16 17:30:48.945198 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:30:48.945175 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r8cn4_f637d961-672d-4aa2-a49a-fd021c2d4df2/console-operator/2.log" Jul 16 17:30:48.946435 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:30:48.946418 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qd7bg_cfbe9f0b-b759-4064-9332-3b48c547c003/ovn-acl-logging/0.log" Jul 16 17:30:48.952866 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:30:48.952846 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qd7bg_cfbe9f0b-b759-4064-9332-3b48c547c003/ovn-acl-logging/0.log" Jul 16 17:31:50.482116 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:31:50.482085 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-b44c0f/rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5"] Jul 16 17:31:50.482633 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:31:50.482472 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2dd6fd20-20ad-4d6d-9970-a2881ed5e74d" containerName="main" Jul 16 17:31:50.482633 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:31:50.482490 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dd6fd20-20ad-4d6d-9970-a2881ed5e74d" containerName="main" Jul 16 17:31:50.482633 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:31:50.482594 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="2dd6fd20-20ad-4d6d-9970-a2881ed5e74d" containerName="main" Jul 16 17:31:50.485539 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:31:50.485523 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b44c0f/rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5" Jul 16 17:31:50.489972 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:31:50.489947 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b44c0f\"/\"default-dockercfg-2r7cx\"" Jul 16 17:31:50.490103 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:31:50.489985 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b44c0f\"/\"openshift-service-ca.crt\"" Jul 16 17:31:50.490103 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:31:50.489953 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b44c0f\"/\"rate-test-model-b44c0f-kserve-self-signed-certs\"" Jul 16 17:31:50.490103 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:31:50.490032 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b44c0f\"/\"kube-root-ca.crt\"" Jul 16 17:31:50.503326 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:31:50.503304 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b44c0f/rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5"] Jul 16 17:31:50.510840 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:31:50.510819 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39b0303f-aa57-47fb-83fb-501338019835-model-cache\") pod \"rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5\" (UID: \"39b0303f-aa57-47fb-83fb-501338019835\") " pod="ai-tenant-e2e-shared-a-b44c0f/rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5" Jul 16 17:31:50.510969 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:31:50.510854 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gg5gd\" (UniqueName: \"kubernetes.io/projected/39b0303f-aa57-47fb-83fb-501338019835-kube-api-access-gg5gd\") pod \"rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5\" (UID: \"39b0303f-aa57-47fb-83fb-501338019835\") " pod="ai-tenant-e2e-shared-a-b44c0f/rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5" Jul 16 17:31:50.510969 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:31:50.510934 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39b0303f-aa57-47fb-83fb-501338019835-home\") pod \"rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5\" (UID: \"39b0303f-aa57-47fb-83fb-501338019835\") " pod="ai-tenant-e2e-shared-a-b44c0f/rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5" Jul 16 17:31:50.510969 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:31:50.510960 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39b0303f-aa57-47fb-83fb-501338019835-tls-certs\") pod \"rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5\" (UID: \"39b0303f-aa57-47fb-83fb-501338019835\") " pod="ai-tenant-e2e-shared-a-b44c0f/rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5" Jul 16 17:31:50.511101 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:31:50.511036 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39b0303f-aa57-47fb-83fb-501338019835-dshm\") pod \"rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5\" (UID: \"39b0303f-aa57-47fb-83fb-501338019835\") " pod="ai-tenant-e2e-shared-a-b44c0f/rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5" Jul 16 17:31:50.511139 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:31:50.511114 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39b0303f-aa57-47fb-83fb-501338019835-tmp-dir\") pod \"rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5\" (UID: \"39b0303f-aa57-47fb-83fb-501338019835\") " pod="ai-tenant-e2e-shared-a-b44c0f/rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5" Jul 16 17:31:50.611871 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:31:50.611834 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39b0303f-aa57-47fb-83fb-501338019835-dshm\") pod \"rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5\" (UID: \"39b0303f-aa57-47fb-83fb-501338019835\") " pod="ai-tenant-e2e-shared-a-b44c0f/rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5" Jul 16 17:31:50.612061 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:31:50.611891 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39b0303f-aa57-47fb-83fb-501338019835-tmp-dir\") pod \"rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5\" (UID: \"39b0303f-aa57-47fb-83fb-501338019835\") " pod="ai-tenant-e2e-shared-a-b44c0f/rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5" Jul 16 17:31:50.612061 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:31:50.611924 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39b0303f-aa57-47fb-83fb-501338019835-model-cache\") pod \"rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5\" (UID: \"39b0303f-aa57-47fb-83fb-501338019835\") " pod="ai-tenant-e2e-shared-a-b44c0f/rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5" Jul 16 17:31:50.612061 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:31:50.611943 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gg5gd\" (UniqueName: \"kubernetes.io/projected/39b0303f-aa57-47fb-83fb-501338019835-kube-api-access-gg5gd\") pod \"rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5\" (UID: \"39b0303f-aa57-47fb-83fb-501338019835\") " pod="ai-tenant-e2e-shared-a-b44c0f/rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5" Jul 16 17:31:50.612061 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:31:50.611980 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39b0303f-aa57-47fb-83fb-501338019835-home\") pod \"rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5\" (UID: \"39b0303f-aa57-47fb-83fb-501338019835\") " pod="ai-tenant-e2e-shared-a-b44c0f/rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5" Jul 16 17:31:50.612061 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:31:50.611994 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39b0303f-aa57-47fb-83fb-501338019835-tls-certs\") pod \"rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5\" (UID: \"39b0303f-aa57-47fb-83fb-501338019835\") " pod="ai-tenant-e2e-shared-a-b44c0f/rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5" Jul 16 17:31:50.612401 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:31:50.612377 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39b0303f-aa57-47fb-83fb-501338019835-tmp-dir\") pod \"rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5\" (UID: \"39b0303f-aa57-47fb-83fb-501338019835\") " pod="ai-tenant-e2e-shared-a-b44c0f/rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5" Jul 16 17:31:50.612442 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:31:50.612388 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39b0303f-aa57-47fb-83fb-501338019835-model-cache\") pod \"rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5\" (UID: \"39b0303f-aa57-47fb-83fb-501338019835\") " pod="ai-tenant-e2e-shared-a-b44c0f/rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5" Jul 16 17:31:50.612442 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:31:50.612413 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39b0303f-aa57-47fb-83fb-501338019835-home\") pod \"rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5\" (UID: \"39b0303f-aa57-47fb-83fb-501338019835\") " pod="ai-tenant-e2e-shared-a-b44c0f/rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5" Jul 16 17:31:50.614260 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:31:50.614238 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39b0303f-aa57-47fb-83fb-501338019835-dshm\") pod \"rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5\" (UID: \"39b0303f-aa57-47fb-83fb-501338019835\") " pod="ai-tenant-e2e-shared-a-b44c0f/rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5" Jul 16 17:31:50.614447 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:31:50.614430 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39b0303f-aa57-47fb-83fb-501338019835-tls-certs\") pod \"rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5\" (UID: \"39b0303f-aa57-47fb-83fb-501338019835\") " pod="ai-tenant-e2e-shared-a-b44c0f/rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5" Jul 16 17:31:50.621407 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:31:50.621387 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gg5gd\" (UniqueName: \"kubernetes.io/projected/39b0303f-aa57-47fb-83fb-501338019835-kube-api-access-gg5gd\") pod \"rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5\" (UID: \"39b0303f-aa57-47fb-83fb-501338019835\") " pod="ai-tenant-e2e-shared-a-b44c0f/rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5" Jul 16 17:31:50.795485 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:31:50.795412 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b44c0f/rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5" Jul 16 17:31:50.946558 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:31:50.946535 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b44c0f/rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5"] Jul 16 17:31:50.948804 ip-10-0-128-50 kubenswrapper[2571]: W0716 17:31:50.948770 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod39b0303f_aa57_47fb_83fb_501338019835.slice/crio-70a82ec0513fd81346a4971598da0d98ed5e07b62e26b516b8108c7ed97b248f WatchSource:0}: Error finding container 70a82ec0513fd81346a4971598da0d98ed5e07b62e26b516b8108c7ed97b248f: Status 404 returned error can't find the container with id 70a82ec0513fd81346a4971598da0d98ed5e07b62e26b516b8108c7ed97b248f Jul 16 17:31:51.339307 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:31:51.339275 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b44c0f/rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5" event={"ID":"39b0303f-aa57-47fb-83fb-501338019835","Type":"ContainerStarted","Data":"d7405216d8544bb7bfe08b748fe462c968d21e1e74c8ad1550a5a29d0811f3b5"} Jul 16 17:31:51.339307 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:31:51.339316 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b44c0f/rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5" event={"ID":"39b0303f-aa57-47fb-83fb-501338019835","Type":"ContainerStarted","Data":"70a82ec0513fd81346a4971598da0d98ed5e07b62e26b516b8108c7ed97b248f"} Jul 16 17:31:51.368133 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:31:51.368088 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-b44c0f/rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5" podStartSLOduration=1.368072874 podStartE2EDuration="1.368072874s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:31:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:31:51.36744922 +0000 UTC m=+2463.109024518" watchObservedRunningTime="2026-07-16 17:31:51.368072874 +0000 UTC m=+2463.109648170" Jul 16 17:32:00.796521 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:32:00.796488 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-b44c0f/rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5" Jul 16 17:32:00.799039 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:32:00.796531 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-b44c0f/rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5" Jul 16 17:32:00.809218 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:32:00.809182 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-b44c0f/rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5" Jul 16 17:32:01.403214 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:32:01.403180 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-b44c0f/rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5" Jul 16 17:32:17.596205 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:32:17.596173 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-8d3a22/rate-test-model-8d3a22-kserve-784d9d9675-7gfgq"] Jul 16 17:32:17.599473 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:32:17.599455 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8d3a22/rate-test-model-8d3a22-kserve-784d9d9675-7gfgq" Jul 16 17:32:17.604035 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:32:17.604011 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8d3a22\"/\"openshift-service-ca.crt\"" Jul 16 17:32:17.605136 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:32:17.605119 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8d3a22\"/\"rate-test-model-8d3a22-kserve-self-signed-certs\"" Jul 16 17:32:17.605238 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:32:17.605118 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8d3a22\"/\"kube-root-ca.crt\"" Jul 16 17:32:17.605238 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:32:17.605170 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8d3a22\"/\"default-dockercfg-4xt5g\"" Jul 16 17:32:17.613450 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:32:17.613426 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8d3a22/rate-test-model-8d3a22-kserve-784d9d9675-7gfgq"] Jul 16 17:32:17.736692 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:32:17.736659 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1009053d-ed5e-4a7e-9dca-037118bd6fcb-dshm\") pod \"rate-test-model-8d3a22-kserve-784d9d9675-7gfgq\" (UID: \"1009053d-ed5e-4a7e-9dca-037118bd6fcb\") " pod="ai-tenant-e2e-shared-b-8d3a22/rate-test-model-8d3a22-kserve-784d9d9675-7gfgq" Jul 16 17:32:17.736881 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:32:17.736706 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxkw2\" (UniqueName: \"kubernetes.io/projected/1009053d-ed5e-4a7e-9dca-037118bd6fcb-kube-api-access-hxkw2\") pod \"rate-test-model-8d3a22-kserve-784d9d9675-7gfgq\" (UID: \"1009053d-ed5e-4a7e-9dca-037118bd6fcb\") " pod="ai-tenant-e2e-shared-b-8d3a22/rate-test-model-8d3a22-kserve-784d9d9675-7gfgq" Jul 16 17:32:17.736881 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:32:17.736821 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1009053d-ed5e-4a7e-9dca-037118bd6fcb-model-cache\") pod \"rate-test-model-8d3a22-kserve-784d9d9675-7gfgq\" (UID: \"1009053d-ed5e-4a7e-9dca-037118bd6fcb\") " pod="ai-tenant-e2e-shared-b-8d3a22/rate-test-model-8d3a22-kserve-784d9d9675-7gfgq" Jul 16 17:32:17.736881 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:32:17.736848 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1009053d-ed5e-4a7e-9dca-037118bd6fcb-tmp-dir\") pod \"rate-test-model-8d3a22-kserve-784d9d9675-7gfgq\" (UID: \"1009053d-ed5e-4a7e-9dca-037118bd6fcb\") " pod="ai-tenant-e2e-shared-b-8d3a22/rate-test-model-8d3a22-kserve-784d9d9675-7gfgq" Jul 16 17:32:17.736991 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:32:17.736884 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1009053d-ed5e-4a7e-9dca-037118bd6fcb-tls-certs\") pod \"rate-test-model-8d3a22-kserve-784d9d9675-7gfgq\" (UID: \"1009053d-ed5e-4a7e-9dca-037118bd6fcb\") " pod="ai-tenant-e2e-shared-b-8d3a22/rate-test-model-8d3a22-kserve-784d9d9675-7gfgq" Jul 16 17:32:17.736991 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:32:17.736923 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1009053d-ed5e-4a7e-9dca-037118bd6fcb-home\") pod \"rate-test-model-8d3a22-kserve-784d9d9675-7gfgq\" (UID: \"1009053d-ed5e-4a7e-9dca-037118bd6fcb\") " pod="ai-tenant-e2e-shared-b-8d3a22/rate-test-model-8d3a22-kserve-784d9d9675-7gfgq" Jul 16 17:32:17.838115 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:32:17.838082 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1009053d-ed5e-4a7e-9dca-037118bd6fcb-home\") pod \"rate-test-model-8d3a22-kserve-784d9d9675-7gfgq\" (UID: \"1009053d-ed5e-4a7e-9dca-037118bd6fcb\") " pod="ai-tenant-e2e-shared-b-8d3a22/rate-test-model-8d3a22-kserve-784d9d9675-7gfgq" Jul 16 17:32:17.838285 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:32:17.838141 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1009053d-ed5e-4a7e-9dca-037118bd6fcb-dshm\") pod \"rate-test-model-8d3a22-kserve-784d9d9675-7gfgq\" (UID: \"1009053d-ed5e-4a7e-9dca-037118bd6fcb\") " pod="ai-tenant-e2e-shared-b-8d3a22/rate-test-model-8d3a22-kserve-784d9d9675-7gfgq" Jul 16 17:32:17.838285 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:32:17.838180 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxkw2\" (UniqueName: \"kubernetes.io/projected/1009053d-ed5e-4a7e-9dca-037118bd6fcb-kube-api-access-hxkw2\") pod \"rate-test-model-8d3a22-kserve-784d9d9675-7gfgq\" (UID: \"1009053d-ed5e-4a7e-9dca-037118bd6fcb\") " pod="ai-tenant-e2e-shared-b-8d3a22/rate-test-model-8d3a22-kserve-784d9d9675-7gfgq" Jul 16 17:32:17.838285 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:32:17.838248 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1009053d-ed5e-4a7e-9dca-037118bd6fcb-model-cache\") pod \"rate-test-model-8d3a22-kserve-784d9d9675-7gfgq\" (UID: \"1009053d-ed5e-4a7e-9dca-037118bd6fcb\") " pod="ai-tenant-e2e-shared-b-8d3a22/rate-test-model-8d3a22-kserve-784d9d9675-7gfgq" Jul 16 17:32:17.838285 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:32:17.838270 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1009053d-ed5e-4a7e-9dca-037118bd6fcb-tmp-dir\") pod \"rate-test-model-8d3a22-kserve-784d9d9675-7gfgq\" (UID: \"1009053d-ed5e-4a7e-9dca-037118bd6fcb\") " pod="ai-tenant-e2e-shared-b-8d3a22/rate-test-model-8d3a22-kserve-784d9d9675-7gfgq" Jul 16 17:32:17.838515 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:32:17.838316 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1009053d-ed5e-4a7e-9dca-037118bd6fcb-tls-certs\") pod \"rate-test-model-8d3a22-kserve-784d9d9675-7gfgq\" (UID: \"1009053d-ed5e-4a7e-9dca-037118bd6fcb\") " pod="ai-tenant-e2e-shared-b-8d3a22/rate-test-model-8d3a22-kserve-784d9d9675-7gfgq" Jul 16 17:32:17.838586 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:32:17.838527 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1009053d-ed5e-4a7e-9dca-037118bd6fcb-home\") pod \"rate-test-model-8d3a22-kserve-784d9d9675-7gfgq\" (UID: \"1009053d-ed5e-4a7e-9dca-037118bd6fcb\") " pod="ai-tenant-e2e-shared-b-8d3a22/rate-test-model-8d3a22-kserve-784d9d9675-7gfgq" Jul 16 17:32:17.839058 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:32:17.838844 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1009053d-ed5e-4a7e-9dca-037118bd6fcb-tmp-dir\") pod \"rate-test-model-8d3a22-kserve-784d9d9675-7gfgq\" (UID: \"1009053d-ed5e-4a7e-9dca-037118bd6fcb\") " pod="ai-tenant-e2e-shared-b-8d3a22/rate-test-model-8d3a22-kserve-784d9d9675-7gfgq" Jul 16 17:32:17.839173 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:32:17.839034 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1009053d-ed5e-4a7e-9dca-037118bd6fcb-model-cache\") pod \"rate-test-model-8d3a22-kserve-784d9d9675-7gfgq\" (UID: \"1009053d-ed5e-4a7e-9dca-037118bd6fcb\") " pod="ai-tenant-e2e-shared-b-8d3a22/rate-test-model-8d3a22-kserve-784d9d9675-7gfgq" Jul 16 17:32:17.841074 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:32:17.841048 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1009053d-ed5e-4a7e-9dca-037118bd6fcb-dshm\") pod \"rate-test-model-8d3a22-kserve-784d9d9675-7gfgq\" (UID: \"1009053d-ed5e-4a7e-9dca-037118bd6fcb\") " pod="ai-tenant-e2e-shared-b-8d3a22/rate-test-model-8d3a22-kserve-784d9d9675-7gfgq" Jul 16 17:32:17.841249 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:32:17.841232 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1009053d-ed5e-4a7e-9dca-037118bd6fcb-tls-certs\") pod \"rate-test-model-8d3a22-kserve-784d9d9675-7gfgq\" (UID: \"1009053d-ed5e-4a7e-9dca-037118bd6fcb\") " pod="ai-tenant-e2e-shared-b-8d3a22/rate-test-model-8d3a22-kserve-784d9d9675-7gfgq" Jul 16 17:32:17.848707 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:32:17.848652 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxkw2\" (UniqueName: \"kubernetes.io/projected/1009053d-ed5e-4a7e-9dca-037118bd6fcb-kube-api-access-hxkw2\") pod \"rate-test-model-8d3a22-kserve-784d9d9675-7gfgq\" (UID: \"1009053d-ed5e-4a7e-9dca-037118bd6fcb\") " pod="ai-tenant-e2e-shared-b-8d3a22/rate-test-model-8d3a22-kserve-784d9d9675-7gfgq" Jul 16 17:32:17.909199 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:32:17.909162 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8d3a22/rate-test-model-8d3a22-kserve-784d9d9675-7gfgq" Jul 16 17:32:18.051679 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:32:18.051647 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8d3a22/rate-test-model-8d3a22-kserve-784d9d9675-7gfgq"] Jul 16 17:32:18.055333 ip-10-0-128-50 kubenswrapper[2571]: W0716 17:32:18.055307 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1009053d_ed5e_4a7e_9dca_037118bd6fcb.slice/crio-df961e207f774a496988800ba07ea41a97b51b5fd566d066536b42d1270b3ac4 WatchSource:0}: Error finding container df961e207f774a496988800ba07ea41a97b51b5fd566d066536b42d1270b3ac4: Status 404 returned error can't find the container with id df961e207f774a496988800ba07ea41a97b51b5fd566d066536b42d1270b3ac4 Jul 16 17:32:18.480489 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:32:18.480455 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8d3a22/rate-test-model-8d3a22-kserve-784d9d9675-7gfgq" event={"ID":"1009053d-ed5e-4a7e-9dca-037118bd6fcb","Type":"ContainerStarted","Data":"54378d11d7149e963b19b82e6879d22b11e74242adc68d2caa34c7d6d6c99177"} Jul 16 17:32:18.480641 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:32:18.480499 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8d3a22/rate-test-model-8d3a22-kserve-784d9d9675-7gfgq" event={"ID":"1009053d-ed5e-4a7e-9dca-037118bd6fcb","Type":"ContainerStarted","Data":"df961e207f774a496988800ba07ea41a97b51b5fd566d066536b42d1270b3ac4"} Jul 16 17:32:18.515857 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:32:18.515808 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-8d3a22/rate-test-model-8d3a22-kserve-784d9d9675-7gfgq" podStartSLOduration=1.5157930560000001 podStartE2EDuration="1.515793056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:32:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:32:18.514121711 +0000 UTC m=+2490.255697008" watchObservedRunningTime="2026-07-16 17:32:18.515793056 +0000 UTC m=+2490.257368378" Jul 16 17:32:27.909600 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:32:27.909568 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-8d3a22/rate-test-model-8d3a22-kserve-784d9d9675-7gfgq" Jul 16 17:32:27.910098 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:32:27.909613 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-8d3a22/rate-test-model-8d3a22-kserve-784d9d9675-7gfgq" Jul 16 17:32:27.921627 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:32:27.921595 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-8d3a22/rate-test-model-8d3a22-kserve-784d9d9675-7gfgq" Jul 16 17:32:28.543767 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:32:28.543740 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-8d3a22/rate-test-model-8d3a22-kserve-784d9d9675-7gfgq" Jul 16 17:33:04.552249 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:04.552154 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b44c0f/rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5"] Jul 16 17:33:05.189878 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:05.189847 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8d3a22/rate-test-model-8d3a22-kserve-784d9d9675-7gfgq"] Jul 16 17:33:19.574344 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:19.574297 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-b44c0f/rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5" podUID="39b0303f-aa57-47fb-83fb-501338019835" containerName="main" containerID="cri-o://d7405216d8544bb7bfe08b748fe462c968d21e1e74c8ad1550a5a29d0811f3b5" gracePeriod=45 Jul 16 17:33:19.810000 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:19.809961 2571 generic.go:356] "Generic (PLEG): container finished" podID="39b0303f-aa57-47fb-83fb-501338019835" containerID="d7405216d8544bb7bfe08b748fe462c968d21e1e74c8ad1550a5a29d0811f3b5" exitCode=0 Jul 16 17:33:19.810181 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:19.810030 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b44c0f/rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5" event={"ID":"39b0303f-aa57-47fb-83fb-501338019835","Type":"ContainerDied","Data":"d7405216d8544bb7bfe08b748fe462c968d21e1e74c8ad1550a5a29d0811f3b5"} Jul 16 17:33:19.830434 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:19.830387 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b44c0f/rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5" Jul 16 17:33:19.953002 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:19.952974 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39b0303f-aa57-47fb-83fb-501338019835-tls-certs\") pod \"39b0303f-aa57-47fb-83fb-501338019835\" (UID: \"39b0303f-aa57-47fb-83fb-501338019835\") " Jul 16 17:33:19.953165 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:19.953010 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39b0303f-aa57-47fb-83fb-501338019835-tmp-dir\") pod \"39b0303f-aa57-47fb-83fb-501338019835\" (UID: \"39b0303f-aa57-47fb-83fb-501338019835\") " Jul 16 17:33:19.953165 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:19.953072 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39b0303f-aa57-47fb-83fb-501338019835-home\") pod \"39b0303f-aa57-47fb-83fb-501338019835\" (UID: \"39b0303f-aa57-47fb-83fb-501338019835\") " Jul 16 17:33:19.953165 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:19.953096 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39b0303f-aa57-47fb-83fb-501338019835-dshm\") pod \"39b0303f-aa57-47fb-83fb-501338019835\" (UID: \"39b0303f-aa57-47fb-83fb-501338019835\") " Jul 16 17:33:19.953165 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:19.953122 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39b0303f-aa57-47fb-83fb-501338019835-model-cache\") pod \"39b0303f-aa57-47fb-83fb-501338019835\" (UID: \"39b0303f-aa57-47fb-83fb-501338019835\") " Jul 16 17:33:19.953353 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:19.953166 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gg5gd\" (UniqueName: \"kubernetes.io/projected/39b0303f-aa57-47fb-83fb-501338019835-kube-api-access-gg5gd\") pod \"39b0303f-aa57-47fb-83fb-501338019835\" (UID: \"39b0303f-aa57-47fb-83fb-501338019835\") " Jul 16 17:33:19.953410 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:19.953374 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39b0303f-aa57-47fb-83fb-501338019835-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "39b0303f-aa57-47fb-83fb-501338019835" (UID: "39b0303f-aa57-47fb-83fb-501338019835"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:33:19.953410 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:19.953385 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39b0303f-aa57-47fb-83fb-501338019835-home" (OuterVolumeSpecName: "home") pod "39b0303f-aa57-47fb-83fb-501338019835" (UID: "39b0303f-aa57-47fb-83fb-501338019835"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:33:19.953883 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:19.953402 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39b0303f-aa57-47fb-83fb-501338019835-model-cache" (OuterVolumeSpecName: "model-cache") pod "39b0303f-aa57-47fb-83fb-501338019835" (UID: "39b0303f-aa57-47fb-83fb-501338019835"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:33:19.953883 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:19.953465 2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39b0303f-aa57-47fb-83fb-501338019835-tmp-dir\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:33:19.953883 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:19.953477 2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39b0303f-aa57-47fb-83fb-501338019835-home\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:33:19.955223 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:19.955205 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39b0303f-aa57-47fb-83fb-501338019835-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "39b0303f-aa57-47fb-83fb-501338019835" (UID: "39b0303f-aa57-47fb-83fb-501338019835"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:33:19.955303 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:19.955281 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39b0303f-aa57-47fb-83fb-501338019835-kube-api-access-gg5gd" (OuterVolumeSpecName: "kube-api-access-gg5gd") pod "39b0303f-aa57-47fb-83fb-501338019835" (UID: "39b0303f-aa57-47fb-83fb-501338019835"). InnerVolumeSpecName "kube-api-access-gg5gd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:33:19.955350 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:19.955304 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39b0303f-aa57-47fb-83fb-501338019835-dshm" (OuterVolumeSpecName: "dshm") pod "39b0303f-aa57-47fb-83fb-501338019835" (UID: "39b0303f-aa57-47fb-83fb-501338019835"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:33:20.054933 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:20.054903 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gg5gd\" (UniqueName: \"kubernetes.io/projected/39b0303f-aa57-47fb-83fb-501338019835-kube-api-access-gg5gd\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:33:20.054933 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:20.054929 2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39b0303f-aa57-47fb-83fb-501338019835-tls-certs\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:33:20.054933 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:20.054940 2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39b0303f-aa57-47fb-83fb-501338019835-dshm\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:33:20.055130 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:20.054948 2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39b0303f-aa57-47fb-83fb-501338019835-model-cache\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:33:20.210958 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:20.210918 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-8d3a22/rate-test-model-8d3a22-kserve-784d9d9675-7gfgq" podUID="1009053d-ed5e-4a7e-9dca-037118bd6fcb" containerName="main" containerID="cri-o://54378d11d7149e963b19b82e6879d22b11e74242adc68d2caa34c7d6d6c99177" gracePeriod=45 Jul 16 17:33:20.447338 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:20.447314 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8d3a22/rate-test-model-8d3a22-kserve-784d9d9675-7gfgq" Jul 16 17:33:20.559981 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:20.559902 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1009053d-ed5e-4a7e-9dca-037118bd6fcb-tmp-dir\") pod \"1009053d-ed5e-4a7e-9dca-037118bd6fcb\" (UID: \"1009053d-ed5e-4a7e-9dca-037118bd6fcb\") " Jul 16 17:33:20.559981 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:20.559953 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1009053d-ed5e-4a7e-9dca-037118bd6fcb-tls-certs\") pod \"1009053d-ed5e-4a7e-9dca-037118bd6fcb\" (UID: \"1009053d-ed5e-4a7e-9dca-037118bd6fcb\") " Jul 16 17:33:20.560203 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:20.560027 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1009053d-ed5e-4a7e-9dca-037118bd6fcb-home\") pod \"1009053d-ed5e-4a7e-9dca-037118bd6fcb\" (UID: \"1009053d-ed5e-4a7e-9dca-037118bd6fcb\") " Jul 16 17:33:20.560203 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:20.560103 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1009053d-ed5e-4a7e-9dca-037118bd6fcb-model-cache\") pod \"1009053d-ed5e-4a7e-9dca-037118bd6fcb\" (UID: \"1009053d-ed5e-4a7e-9dca-037118bd6fcb\") " Jul 16 17:33:20.560203 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:20.560148 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hxkw2\" (UniqueName: \"kubernetes.io/projected/1009053d-ed5e-4a7e-9dca-037118bd6fcb-kube-api-access-hxkw2\") pod \"1009053d-ed5e-4a7e-9dca-037118bd6fcb\" (UID: \"1009053d-ed5e-4a7e-9dca-037118bd6fcb\") " Jul 16 17:33:20.560203 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:20.560181 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1009053d-ed5e-4a7e-9dca-037118bd6fcb-dshm\") pod \"1009053d-ed5e-4a7e-9dca-037118bd6fcb\" (UID: \"1009053d-ed5e-4a7e-9dca-037118bd6fcb\") " Jul 16 17:33:20.560443 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:20.560211 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1009053d-ed5e-4a7e-9dca-037118bd6fcb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1009053d-ed5e-4a7e-9dca-037118bd6fcb" (UID: "1009053d-ed5e-4a7e-9dca-037118bd6fcb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:33:20.560443 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:20.560246 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1009053d-ed5e-4a7e-9dca-037118bd6fcb-home" (OuterVolumeSpecName: "home") pod "1009053d-ed5e-4a7e-9dca-037118bd6fcb" (UID: "1009053d-ed5e-4a7e-9dca-037118bd6fcb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:33:20.560443 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:20.560370 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1009053d-ed5e-4a7e-9dca-037118bd6fcb-model-cache" (OuterVolumeSpecName: "model-cache") pod "1009053d-ed5e-4a7e-9dca-037118bd6fcb" (UID: "1009053d-ed5e-4a7e-9dca-037118bd6fcb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:33:20.560580 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:20.560473 2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1009053d-ed5e-4a7e-9dca-037118bd6fcb-model-cache\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:33:20.560580 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:20.560491 2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1009053d-ed5e-4a7e-9dca-037118bd6fcb-tmp-dir\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:33:20.560580 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:20.560503 2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1009053d-ed5e-4a7e-9dca-037118bd6fcb-home\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:33:20.561988 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:20.561960 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1009053d-ed5e-4a7e-9dca-037118bd6fcb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1009053d-ed5e-4a7e-9dca-037118bd6fcb" (UID: "1009053d-ed5e-4a7e-9dca-037118bd6fcb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:33:20.562102 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:20.562084 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1009053d-ed5e-4a7e-9dca-037118bd6fcb-dshm" (OuterVolumeSpecName: "dshm") pod "1009053d-ed5e-4a7e-9dca-037118bd6fcb" (UID: "1009053d-ed5e-4a7e-9dca-037118bd6fcb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:33:20.562156 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:20.562132 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1009053d-ed5e-4a7e-9dca-037118bd6fcb-kube-api-access-hxkw2" (OuterVolumeSpecName: "kube-api-access-hxkw2") pod "1009053d-ed5e-4a7e-9dca-037118bd6fcb" (UID: "1009053d-ed5e-4a7e-9dca-037118bd6fcb"). InnerVolumeSpecName "kube-api-access-hxkw2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:33:20.661126 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:20.661099 2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1009053d-ed5e-4a7e-9dca-037118bd6fcb-tls-certs\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:33:20.661126 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:20.661124 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxkw2\" (UniqueName: \"kubernetes.io/projected/1009053d-ed5e-4a7e-9dca-037118bd6fcb-kube-api-access-hxkw2\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:33:20.661466 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:20.661133 2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1009053d-ed5e-4a7e-9dca-037118bd6fcb-dshm\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:33:20.816108 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:20.816047 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b44c0f/rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5" Jul 16 17:33:20.816234 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:20.816047 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b44c0f/rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5" event={"ID":"39b0303f-aa57-47fb-83fb-501338019835","Type":"ContainerDied","Data":"70a82ec0513fd81346a4971598da0d98ed5e07b62e26b516b8108c7ed97b248f"} Jul 16 17:33:20.816234 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:20.816172 2571 scope.go:117] "RemoveContainer" containerID="d7405216d8544bb7bfe08b748fe462c968d21e1e74c8ad1550a5a29d0811f3b5" Jul 16 17:33:20.817480 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:20.817456 2571 generic.go:356] "Generic (PLEG): container finished" podID="1009053d-ed5e-4a7e-9dca-037118bd6fcb" containerID="54378d11d7149e963b19b82e6879d22b11e74242adc68d2caa34c7d6d6c99177" exitCode=0 Jul 16 17:33:20.817560 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:20.817493 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8d3a22/rate-test-model-8d3a22-kserve-784d9d9675-7gfgq" event={"ID":"1009053d-ed5e-4a7e-9dca-037118bd6fcb","Type":"ContainerDied","Data":"54378d11d7149e963b19b82e6879d22b11e74242adc68d2caa34c7d6d6c99177"} Jul 16 17:33:20.817560 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:20.817527 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8d3a22/rate-test-model-8d3a22-kserve-784d9d9675-7gfgq" Jul 16 17:33:20.817560 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:20.817543 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8d3a22/rate-test-model-8d3a22-kserve-784d9d9675-7gfgq" event={"ID":"1009053d-ed5e-4a7e-9dca-037118bd6fcb","Type":"ContainerDied","Data":"df961e207f774a496988800ba07ea41a97b51b5fd566d066536b42d1270b3ac4"} Jul 16 17:33:20.825067 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:20.825052 2571 scope.go:117] "RemoveContainer" containerID="54378d11d7149e963b19b82e6879d22b11e74242adc68d2caa34c7d6d6c99177" Jul 16 17:33:20.832670 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:20.832655 2571 scope.go:117] "RemoveContainer" containerID="54378d11d7149e963b19b82e6879d22b11e74242adc68d2caa34c7d6d6c99177" Jul 16 17:33:20.832919 ip-10-0-128-50 kubenswrapper[2571]: E0716 17:33:20.832900 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54378d11d7149e963b19b82e6879d22b11e74242adc68d2caa34c7d6d6c99177\": container with ID starting with 54378d11d7149e963b19b82e6879d22b11e74242adc68d2caa34c7d6d6c99177 not found: ID does not exist" containerID="54378d11d7149e963b19b82e6879d22b11e74242adc68d2caa34c7d6d6c99177" Jul 16 17:33:20.832971 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:20.832926 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54378d11d7149e963b19b82e6879d22b11e74242adc68d2caa34c7d6d6c99177"} err="failed to get container status \"54378d11d7149e963b19b82e6879d22b11e74242adc68d2caa34c7d6d6c99177\": rpc error: code = NotFound desc = could not find container \"54378d11d7149e963b19b82e6879d22b11e74242adc68d2caa34c7d6d6c99177\": container with ID starting with 54378d11d7149e963b19b82e6879d22b11e74242adc68d2caa34c7d6d6c99177 not found: ID does not exist" Jul 16 17:33:20.838246 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:20.838224 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b44c0f/rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5"] Jul 16 17:33:20.845346 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:20.845326 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-b44c0f/rate-test-model-b44c0f-kserve-7d9b9d5fdc-7g2d5"] Jul 16 17:33:20.859508 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:20.859487 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8d3a22/rate-test-model-8d3a22-kserve-784d9d9675-7gfgq"] Jul 16 17:33:20.861820 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:20.861801 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-8d3a22/rate-test-model-8d3a22-kserve-784d9d9675-7gfgq"] Jul 16 17:33:21.784723 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:21.784691 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7596-140535666843984"] Jul 16 17:33:21.785101 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:21.785033 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1009053d-ed5e-4a7e-9dca-037118bd6fcb" containerName="main" Jul 16 17:33:21.785101 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:21.785044 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1009053d-ed5e-4a7e-9dca-037118bd6fcb" containerName="main" Jul 16 17:33:21.785101 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:21.785065 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39b0303f-aa57-47fb-83fb-501338019835" containerName="main" Jul 16 17:33:21.785101 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:21.785070 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="39b0303f-aa57-47fb-83fb-501338019835" containerName="main" Jul 16 17:33:21.785242 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:21.785132 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="1009053d-ed5e-4a7e-9dca-037118bd6fcb" containerName="main" Jul 16 17:33:21.785242 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:21.785144 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="39b0303f-aa57-47fb-83fb-501338019835" containerName="main" Jul 16 17:33:21.789404 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:21.789377 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7596-140535666843984" Jul 16 17:33:21.792298 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:21.792282 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-lnlsz\"" Jul 16 17:33:21.796428 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:21.796410 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7596-140535666843984"] Jul 16 17:33:21.869785 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:21.869757 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tg6p\" (UniqueName: \"kubernetes.io/projected/fd86341c-c854-4869-ad56-4a3c9a807ef5-kube-api-access-8tg6p\") pod \"test-curl-7596-140535666843984\" (UID: \"fd86341c-c854-4869-ad56-4a3c9a807ef5\") " pod="opendatahub/test-curl-7596-140535666843984" Jul 16 17:33:21.970707 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:21.970682 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tg6p\" (UniqueName: \"kubernetes.io/projected/fd86341c-c854-4869-ad56-4a3c9a807ef5-kube-api-access-8tg6p\") pod \"test-curl-7596-140535666843984\" (UID: \"fd86341c-c854-4869-ad56-4a3c9a807ef5\") " pod="opendatahub/test-curl-7596-140535666843984" Jul 16 17:33:21.981192 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:21.981161 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tg6p\" (UniqueName: \"kubernetes.io/projected/fd86341c-c854-4869-ad56-4a3c9a807ef5-kube-api-access-8tg6p\") pod \"test-curl-7596-140535666843984\" (UID: \"fd86341c-c854-4869-ad56-4a3c9a807ef5\") " pod="opendatahub/test-curl-7596-140535666843984" Jul 16 17:33:22.098445 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:22.098386 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7596-140535666843984" Jul 16 17:33:22.238310 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:22.238281 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7596-140535666843984"] Jul 16 17:33:22.240326 ip-10-0-128-50 kubenswrapper[2571]: W0716 17:33:22.240297 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfd86341c_c854_4869_ad56_4a3c9a807ef5.slice/crio-15d77633b96d2d95515d6479deb35de829a8b9ba0ab7cd80b705a0c2378c4766 WatchSource:0}: Error finding container 15d77633b96d2d95515d6479deb35de829a8b9ba0ab7cd80b705a0c2378c4766: Status 404 returned error can't find the container with id 15d77633b96d2d95515d6479deb35de829a8b9ba0ab7cd80b705a0c2378c4766 Jul 16 17:33:22.241914 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:22.241897 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 17:33:22.779016 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:22.778968 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1009053d-ed5e-4a7e-9dca-037118bd6fcb" path="/var/lib/kubelet/pods/1009053d-ed5e-4a7e-9dca-037118bd6fcb/volumes" Jul 16 17:33:22.779413 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:22.779393 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39b0303f-aa57-47fb-83fb-501338019835" path="/var/lib/kubelet/pods/39b0303f-aa57-47fb-83fb-501338019835/volumes" Jul 16 17:33:22.835053 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:22.835019 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7596-140535666843984" event={"ID":"fd86341c-c854-4869-ad56-4a3c9a807ef5","Type":"ContainerStarted","Data":"15d77633b96d2d95515d6479deb35de829a8b9ba0ab7cd80b705a0c2378c4766"} Jul 16 17:33:23.841613 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:23.841578 2571 generic.go:356] "Generic (PLEG): container finished" podID="fd86341c-c854-4869-ad56-4a3c9a807ef5" containerID="ecb8f7323a9c784ee0da96a4a06b0917106a830a44a74ff71fdbf3317e184b6d" exitCode=0 Jul 16 17:33:23.842018 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:23.841673 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7596-140535666843984" event={"ID":"fd86341c-c854-4869-ad56-4a3c9a807ef5","Type":"ContainerDied","Data":"ecb8f7323a9c784ee0da96a4a06b0917106a830a44a74ff71fdbf3317e184b6d"} Jul 16 17:33:24.977379 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:24.977354 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7596-140535666843984" Jul 16 17:33:25.028746 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:25.028701 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7596-140535666843984_fd86341c-c854-4869-ad56-4a3c9a807ef5/test-curl-7596-140535666843984/0.log" Jul 16 17:33:25.064083 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:25.064050 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7596-140535666843984"] Jul 16 17:33:25.069713 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:25.069689 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7596-140535666843984"] Jul 16 17:33:25.098340 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:25.098283 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8tg6p\" (UniqueName: \"kubernetes.io/projected/fd86341c-c854-4869-ad56-4a3c9a807ef5-kube-api-access-8tg6p\") pod \"fd86341c-c854-4869-ad56-4a3c9a807ef5\" (UID: \"fd86341c-c854-4869-ad56-4a3c9a807ef5\") " Jul 16 17:33:25.100368 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:25.100345 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd86341c-c854-4869-ad56-4a3c9a807ef5-kube-api-access-8tg6p" (OuterVolumeSpecName: "kube-api-access-8tg6p") pod "fd86341c-c854-4869-ad56-4a3c9a807ef5" (UID: "fd86341c-c854-4869-ad56-4a3c9a807ef5"). InnerVolumeSpecName "kube-api-access-8tg6p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:33:25.199474 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:25.199446 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8tg6p\" (UniqueName: \"kubernetes.io/projected/fd86341c-c854-4869-ad56-4a3c9a807ef5-kube-api-access-8tg6p\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:33:25.852742 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:25.852708 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7596-140535666843984" Jul 16 17:33:25.852913 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:25.852716 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="15d77633b96d2d95515d6479deb35de829a8b9ba0ab7cd80b705a0c2378c4766" Jul 16 17:33:25.863364 ip-10-0-128-50 kubenswrapper[2571]: E0716 17:33:25.863335 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7596-140535666843984\" is forbidden: User \"system:node:ip-10-0-128-50.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-50.ec2.internal' and this object" podUID="fd86341c-c854-4869-ad56-4a3c9a807ef5" pod="opendatahub/test-curl-7596-140535666843984" Jul 16 17:33:26.777412 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:26.777373 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd86341c-c854-4869-ad56-4a3c9a807ef5" path="/var/lib/kubelet/pods/fd86341c-c854-4869-ad56-4a3c9a807ef5/volumes" Jul 16 17:33:40.317178 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:40.317136 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7"] Jul 16 17:33:40.317688 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:40.317363 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" podUID="82e15968-cfde-4887-97c5-33719f95bf87" containerName="istio-proxy" containerID="cri-o://082f3293d925dbfc517408647b98ab19cbae24c19a030a8814faf316b220766d" gracePeriod=30 Jul 16 17:33:43.292496 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:43.292460 2571 patch_prober.go:28] interesting pod/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 16 17:33:43.292887 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:43.292527 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" podUID="82e15968-cfde-4887-97c5-33719f95bf87" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 17:33:45.573976 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.573949 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" Jul 16 17:33:45.663569 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.663500 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xbf9s\" (UniqueName: \"kubernetes.io/projected/82e15968-cfde-4887-97c5-33719f95bf87-kube-api-access-xbf9s\") pod \"82e15968-cfde-4887-97c5-33719f95bf87\" (UID: \"82e15968-cfde-4887-97c5-33719f95bf87\") " Jul 16 17:33:45.663569 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.663539 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82e15968-cfde-4887-97c5-33719f95bf87-istio-podinfo\") pod \"82e15968-cfde-4887-97c5-33719f95bf87\" (UID: \"82e15968-cfde-4887-97c5-33719f95bf87\") " Jul 16 17:33:45.663569 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.663570 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82e15968-cfde-4887-97c5-33719f95bf87-workload-certs\") pod \"82e15968-cfde-4887-97c5-33719f95bf87\" (UID: \"82e15968-cfde-4887-97c5-33719f95bf87\") " Jul 16 17:33:45.663832 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.663586 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82e15968-cfde-4887-97c5-33719f95bf87-istio-token\") pod \"82e15968-cfde-4887-97c5-33719f95bf87\" (UID: \"82e15968-cfde-4887-97c5-33719f95bf87\") " Jul 16 17:33:45.663832 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.663609 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82e15968-cfde-4887-97c5-33719f95bf87-istiod-ca-cert\") pod \"82e15968-cfde-4887-97c5-33719f95bf87\" (UID: \"82e15968-cfde-4887-97c5-33719f95bf87\") " Jul 16 17:33:45.663832 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.663656 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82e15968-cfde-4887-97c5-33719f95bf87-istio-envoy\") pod \"82e15968-cfde-4887-97c5-33719f95bf87\" (UID: \"82e15968-cfde-4887-97c5-33719f95bf87\") " Jul 16 17:33:45.663832 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.663681 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82e15968-cfde-4887-97c5-33719f95bf87-workload-socket\") pod \"82e15968-cfde-4887-97c5-33719f95bf87\" (UID: \"82e15968-cfde-4887-97c5-33719f95bf87\") " Jul 16 17:33:45.663832 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.663777 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82e15968-cfde-4887-97c5-33719f95bf87-istio-data\") pod \"82e15968-cfde-4887-97c5-33719f95bf87\" (UID: \"82e15968-cfde-4887-97c5-33719f95bf87\") " Jul 16 17:33:45.663832 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.663800 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82e15968-cfde-4887-97c5-33719f95bf87-credential-socket\") pod \"82e15968-cfde-4887-97c5-33719f95bf87\" (UID: \"82e15968-cfde-4887-97c5-33719f95bf87\") " Jul 16 17:33:45.664143 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.663910 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82e15968-cfde-4887-97c5-33719f95bf87-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "82e15968-cfde-4887-97c5-33719f95bf87" (UID: "82e15968-cfde-4887-97c5-33719f95bf87"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:33:45.664143 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.664088 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82e15968-cfde-4887-97c5-33719f95bf87-workload-certs\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:33:45.664250 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.664169 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82e15968-cfde-4887-97c5-33719f95bf87-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "82e15968-cfde-4887-97c5-33719f95bf87" (UID: "82e15968-cfde-4887-97c5-33719f95bf87"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:33:45.664302 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.664270 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82e15968-cfde-4887-97c5-33719f95bf87-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "82e15968-cfde-4887-97c5-33719f95bf87" (UID: "82e15968-cfde-4887-97c5-33719f95bf87"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:33:45.664302 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.664279 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82e15968-cfde-4887-97c5-33719f95bf87-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "82e15968-cfde-4887-97c5-33719f95bf87" (UID: "82e15968-cfde-4887-97c5-33719f95bf87"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:33:45.664765 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.664718 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82e15968-cfde-4887-97c5-33719f95bf87-istio-data" (OuterVolumeSpecName: "istio-data") pod "82e15968-cfde-4887-97c5-33719f95bf87" (UID: "82e15968-cfde-4887-97c5-33719f95bf87"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:33:45.665842 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.665816 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/82e15968-cfde-4887-97c5-33719f95bf87-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "82e15968-cfde-4887-97c5-33719f95bf87" (UID: "82e15968-cfde-4887-97c5-33719f95bf87"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 17:33:45.665948 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.665880 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82e15968-cfde-4887-97c5-33719f95bf87-istio-token" (OuterVolumeSpecName: "istio-token") pod "82e15968-cfde-4887-97c5-33719f95bf87" (UID: "82e15968-cfde-4887-97c5-33719f95bf87"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:33:45.665948 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.665895 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82e15968-cfde-4887-97c5-33719f95bf87-kube-api-access-xbf9s" (OuterVolumeSpecName: "kube-api-access-xbf9s") pod "82e15968-cfde-4887-97c5-33719f95bf87" (UID: "82e15968-cfde-4887-97c5-33719f95bf87"). InnerVolumeSpecName "kube-api-access-xbf9s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:33:45.666052 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.666036 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82e15968-cfde-4887-97c5-33719f95bf87-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "82e15968-cfde-4887-97c5-33719f95bf87" (UID: "82e15968-cfde-4887-97c5-33719f95bf87"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:33:45.764808 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.764787 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82e15968-cfde-4887-97c5-33719f95bf87-istio-envoy\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:33:45.764808 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.764806 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82e15968-cfde-4887-97c5-33719f95bf87-workload-socket\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:33:45.764939 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.764815 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82e15968-cfde-4887-97c5-33719f95bf87-istio-data\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:33:45.764939 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.764823 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82e15968-cfde-4887-97c5-33719f95bf87-credential-socket\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:33:45.764939 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.764832 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xbf9s\" (UniqueName: \"kubernetes.io/projected/82e15968-cfde-4887-97c5-33719f95bf87-kube-api-access-xbf9s\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:33:45.764939 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.764841 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82e15968-cfde-4887-97c5-33719f95bf87-istio-podinfo\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:33:45.764939 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.764849 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82e15968-cfde-4887-97c5-33719f95bf87-istio-token\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:33:45.764939 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.764856 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82e15968-cfde-4887-97c5-33719f95bf87-istiod-ca-cert\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 16 17:33:45.957588 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.957561 2571 generic.go:356] "Generic (PLEG): container finished" podID="82e15968-cfde-4887-97c5-33719f95bf87" containerID="082f3293d925dbfc517408647b98ab19cbae24c19a030a8814faf316b220766d" exitCode=0 Jul 16 17:33:45.957705 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.957611 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" Jul 16 17:33:45.957705 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.957637 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" event={"ID":"82e15968-cfde-4887-97c5-33719f95bf87","Type":"ContainerDied","Data":"082f3293d925dbfc517408647b98ab19cbae24c19a030a8814faf316b220766d"} Jul 16 17:33:45.957705 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.957682 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7" event={"ID":"82e15968-cfde-4887-97c5-33719f95bf87","Type":"ContainerDied","Data":"6bb9c3becded1afa08e5f9c637ac57abb328fcf1e55c58556d4da3b2e9503faf"} Jul 16 17:33:45.957705 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.957705 2571 scope.go:117] "RemoveContainer" containerID="082f3293d925dbfc517408647b98ab19cbae24c19a030a8814faf316b220766d" Jul 16 17:33:45.967184 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.967167 2571 scope.go:117] "RemoveContainer" containerID="78816b218a2e5de77733f805be121be06f0cfb658ce55934081e3a848c206463" Jul 16 17:33:45.974916 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.974902 2571 scope.go:117] "RemoveContainer" containerID="082f3293d925dbfc517408647b98ab19cbae24c19a030a8814faf316b220766d" Jul 16 17:33:45.975135 ip-10-0-128-50 kubenswrapper[2571]: E0716 17:33:45.975118 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"082f3293d925dbfc517408647b98ab19cbae24c19a030a8814faf316b220766d\": container with ID starting with 082f3293d925dbfc517408647b98ab19cbae24c19a030a8814faf316b220766d not found: ID does not exist" containerID="082f3293d925dbfc517408647b98ab19cbae24c19a030a8814faf316b220766d" Jul 16 17:33:45.975177 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.975144 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"082f3293d925dbfc517408647b98ab19cbae24c19a030a8814faf316b220766d"} err="failed to get container status \"082f3293d925dbfc517408647b98ab19cbae24c19a030a8814faf316b220766d\": rpc error: code = NotFound desc = could not find container \"082f3293d925dbfc517408647b98ab19cbae24c19a030a8814faf316b220766d\": container with ID starting with 082f3293d925dbfc517408647b98ab19cbae24c19a030a8814faf316b220766d not found: ID does not exist" Jul 16 17:33:45.975177 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.975159 2571 scope.go:117] "RemoveContainer" containerID="78816b218a2e5de77733f805be121be06f0cfb658ce55934081e3a848c206463" Jul 16 17:33:45.975354 ip-10-0-128-50 kubenswrapper[2571]: E0716 17:33:45.975339 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78816b218a2e5de77733f805be121be06f0cfb658ce55934081e3a848c206463\": container with ID starting with 78816b218a2e5de77733f805be121be06f0cfb658ce55934081e3a848c206463 not found: ID does not exist" containerID="78816b218a2e5de77733f805be121be06f0cfb658ce55934081e3a848c206463" Jul 16 17:33:45.975390 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.975361 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78816b218a2e5de77733f805be121be06f0cfb658ce55934081e3a848c206463"} err="failed to get container status \"78816b218a2e5de77733f805be121be06f0cfb658ce55934081e3a848c206463\": rpc error: code = NotFound desc = could not find container \"78816b218a2e5de77733f805be121be06f0cfb658ce55934081e3a848c206463\": container with ID starting with 78816b218a2e5de77733f805be121be06f0cfb658ce55934081e3a848c206463 not found: ID does not exist" Jul 16 17:33:45.981982 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.981963 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7"] Jul 16 17:33:45.986691 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:45.986668 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-b44c0f-openshift-default-99f688cbc-46zd7"] Jul 16 17:33:46.777878 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:46.777841 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82e15968-cfde-4887-97c5-33719f95bf87" path="/var/lib/kubelet/pods/82e15968-cfde-4887-97c5-33719f95bf87/volumes" Jul 16 17:33:55.858484 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:33:55.858446 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-657df94667-8qzkj_7332c6ef-7c0f-4f06-9e27-f526a138a28b/authorino/0.log" Jul 16 17:34:01.660689 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:01.660657 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-7zf2x_b162a8ad-e02f-4bb7-969a-b27787efcf7b/manager/0.log" Jul 16 17:34:01.791688 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:01.791657 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-85csp_28df4feb-0098-4de5-82c3-d983879f0f41/manager/0.log" Jul 16 17:34:02.044805 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:02.044781 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-l5d8m_581fb7d4-c35a-47cd-9d2f-78dce3928a1a/server/0.log" Jul 16 17:34:02.413069 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:02.412986 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c7f465c76-fwvtz_d4b8d3c4-3c0c-4fb9-9ad0-a3af9bd3495f/copy-manifests/0.log" Jul 16 17:34:02.423431 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:02.423406 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c7f465c76-fwvtz_d4b8d3c4-3c0c-4fb9-9ad0-a3af9bd3495f/manager/0.log" Jul 16 17:34:04.163692 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:04.163668 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75998hh8_4ed08f90-c844-43f1-9c65-ab8bede0ee73/util/0.log" Jul 16 17:34:04.181307 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:04.181287 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75998hh8_4ed08f90-c844-43f1-9c65-ab8bede0ee73/pull/0.log" Jul 16 17:34:04.196612 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:04.196589 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75998hh8_4ed08f90-c844-43f1-9c65-ab8bede0ee73/extract/0.log" Jul 16 17:34:04.315484 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:04.315465 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09psp4_04adff86-9c0b-495e-b73c-29da5b579fda/util/0.log" Jul 16 17:34:04.324868 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:04.324846 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09psp4_04adff86-9c0b-495e-b73c-29da5b579fda/pull/0.log" Jul 16 17:34:04.333846 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:04.333829 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09psp4_04adff86-9c0b-495e-b73c-29da5b579fda/extract/0.log" Jul 16 17:34:04.446288 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:04.446266 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rk7h4_a9b19b46-70a5-46be-b697-dcd420eed74c/util/0.log" Jul 16 17:34:04.456454 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:04.456437 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rk7h4_a9b19b46-70a5-46be-b697-dcd420eed74c/pull/0.log" Jul 16 17:34:04.464827 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:04.464809 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rk7h4_a9b19b46-70a5-46be-b697-dcd420eed74c/extract/0.log" Jul 16 17:34:04.582438 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:04.582415 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pfl45_8df978d6-8d77-40cf-a714-af26edb260f8/util/0.log" Jul 16 17:34:04.590973 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:04.590948 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pfl45_8df978d6-8d77-40cf-a714-af26edb260f8/pull/0.log" Jul 16 17:34:04.602018 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:04.601990 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pfl45_8df978d6-8d77-40cf-a714-af26edb260f8/extract/0.log" Jul 16 17:34:04.731190 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:04.731100 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-657df94667-8qzkj_7332c6ef-7c0f-4f06-9e27-f526a138a28b/authorino/0.log" Jul 16 17:34:04.857021 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:04.856988 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-ck96v_14df11f0-5fc9-491e-8e4a-54867cb706b0/manager/0.log" Jul 16 17:34:05.361965 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:05.361938 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-42p8f_238dde27-77a3-45a2-9ba4-ccc72f0bf58b/manager/0.log" Jul 16 17:34:06.909003 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:06.908977 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-57bc6f48c6-kt2wd_d2bdea19-f5df-4297-8304-da19a8cc7a62/router/0.log" Jul 16 17:34:07.717439 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:07.717408 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2bw6k_cc213918-a6b8-4a89-ac2e-d32e05fc6ae4/main/0.log" Jul 16 17:34:07.953355 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:07.953328 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-lnfs6_0253c5d2-d675-4b8c-b123-9f5da2a8ae3b/main/0.log" Jul 16 17:34:13.812201 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:13.812119 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-p8fjr/must-gather-n8xqz"] Jul 16 17:34:13.812617 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:13.812474 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82e15968-cfde-4887-97c5-33719f95bf87" containerName="istio-proxy" Jul 16 17:34:13.812617 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:13.812487 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="82e15968-cfde-4887-97c5-33719f95bf87" containerName="istio-proxy" Jul 16 17:34:13.812617 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:13.812498 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82e15968-cfde-4887-97c5-33719f95bf87" containerName="istio-proxy" Jul 16 17:34:13.812617 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:13.812504 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="82e15968-cfde-4887-97c5-33719f95bf87" containerName="istio-proxy" Jul 16 17:34:13.812617 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:13.812515 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd86341c-c854-4869-ad56-4a3c9a807ef5" containerName="test-curl-7596-140535666843984" Jul 16 17:34:13.812617 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:13.812522 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd86341c-c854-4869-ad56-4a3c9a807ef5" containerName="test-curl-7596-140535666843984" Jul 16 17:34:13.812617 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:13.812587 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="82e15968-cfde-4887-97c5-33719f95bf87" containerName="istio-proxy" Jul 16 17:34:13.812617 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:13.812595 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd86341c-c854-4869-ad56-4a3c9a807ef5" containerName="test-curl-7596-140535666843984" Jul 16 17:34:13.812617 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:13.812605 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="82e15968-cfde-4887-97c5-33719f95bf87" containerName="istio-proxy" Jul 16 17:34:13.815802 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:13.815782 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-p8fjr/must-gather-n8xqz" Jul 16 17:34:13.821272 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:13.821245 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-p8fjr\"/\"kube-root-ca.crt\"" Jul 16 17:34:13.821633 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:13.821606 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-p8fjr\"/\"openshift-service-ca.crt\"" Jul 16 17:34:13.822803 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:13.822499 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-p8fjr\"/\"default-dockercfg-nbfnt\"" Jul 16 17:34:13.834993 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:13.834967 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-p8fjr/must-gather-n8xqz"] Jul 16 17:34:13.870837 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:13.870804 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qs9c\" (UniqueName: \"kubernetes.io/projected/3eb541bd-0272-4af1-9090-e1fb704041f4-kube-api-access-9qs9c\") pod \"must-gather-n8xqz\" (UID: \"3eb541bd-0272-4af1-9090-e1fb704041f4\") " pod="openshift-must-gather-p8fjr/must-gather-n8xqz" Jul 16 17:34:13.871012 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:13.870891 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3eb541bd-0272-4af1-9090-e1fb704041f4-must-gather-output\") pod \"must-gather-n8xqz\" (UID: \"3eb541bd-0272-4af1-9090-e1fb704041f4\") " pod="openshift-must-gather-p8fjr/must-gather-n8xqz" Jul 16 17:34:13.971652 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:13.971619 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3eb541bd-0272-4af1-9090-e1fb704041f4-must-gather-output\") pod \"must-gather-n8xqz\" (UID: \"3eb541bd-0272-4af1-9090-e1fb704041f4\") " pod="openshift-must-gather-p8fjr/must-gather-n8xqz" Jul 16 17:34:13.971860 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:13.971685 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qs9c\" (UniqueName: \"kubernetes.io/projected/3eb541bd-0272-4af1-9090-e1fb704041f4-kube-api-access-9qs9c\") pod \"must-gather-n8xqz\" (UID: \"3eb541bd-0272-4af1-9090-e1fb704041f4\") " pod="openshift-must-gather-p8fjr/must-gather-n8xqz" Jul 16 17:34:13.971993 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:13.971974 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3eb541bd-0272-4af1-9090-e1fb704041f4-must-gather-output\") pod \"must-gather-n8xqz\" (UID: \"3eb541bd-0272-4af1-9090-e1fb704041f4\") " pod="openshift-must-gather-p8fjr/must-gather-n8xqz" Jul 16 17:34:13.982051 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:13.982025 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qs9c\" (UniqueName: \"kubernetes.io/projected/3eb541bd-0272-4af1-9090-e1fb704041f4-kube-api-access-9qs9c\") pod \"must-gather-n8xqz\" (UID: \"3eb541bd-0272-4af1-9090-e1fb704041f4\") " pod="openshift-must-gather-p8fjr/must-gather-n8xqz" Jul 16 17:34:14.125633 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:14.125555 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-p8fjr/must-gather-n8xqz" Jul 16 17:34:14.274357 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:14.274209 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-p8fjr/must-gather-n8xqz"] Jul 16 17:34:14.277109 ip-10-0-128-50 kubenswrapper[2571]: W0716 17:34:14.277081 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3eb541bd_0272_4af1_9090_e1fb704041f4.slice/crio-68ec8f90b7c7f28a6f7d122b6e7896d5848bb835aa81099558e405eeb404d0b6 WatchSource:0}: Error finding container 68ec8f90b7c7f28a6f7d122b6e7896d5848bb835aa81099558e405eeb404d0b6: Status 404 returned error can't find the container with id 68ec8f90b7c7f28a6f7d122b6e7896d5848bb835aa81099558e405eeb404d0b6 Jul 16 17:34:15.130327 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:15.130301 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-p8fjr/must-gather-n8xqz" event={"ID":"3eb541bd-0272-4af1-9090-e1fb704041f4","Type":"ContainerStarted","Data":"aa2525f861e050ccfda69fa66e9fdf9942dd313d9d5fcd7c61018b8ca9f6f68f"} Jul 16 17:34:15.130627 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:15.130350 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-p8fjr/must-gather-n8xqz" event={"ID":"3eb541bd-0272-4af1-9090-e1fb704041f4","Type":"ContainerStarted","Data":"68ec8f90b7c7f28a6f7d122b6e7896d5848bb835aa81099558e405eeb404d0b6"} Jul 16 17:34:16.140588 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:16.140549 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-p8fjr/must-gather-n8xqz" event={"ID":"3eb541bd-0272-4af1-9090-e1fb704041f4","Type":"ContainerStarted","Data":"6283a40393ae21bced3eb34be4828352d95f605507ce669d4bee10e7629336fb"} Jul 16 17:34:16.165985 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:16.165935 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-p8fjr/must-gather-n8xqz" podStartSLOduration=2.408054177 podStartE2EDuration="3.165921609s" totalImagesPullingTime="757.867432ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:34:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:34:16.164518257 +0000 UTC m=+2607.906093566" watchObservedRunningTime="2026-07-16 17:34:16.165921609 +0000 UTC m=+2607.907496905" Jul 16 17:34:17.168318 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:17.168284 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-wd65t_598a7966-d69c-4867-9936-667f0c47ed23/global-pull-secret-syncer/0.log" Jul 16 17:34:17.324776 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:17.324719 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-df9xm_27c7185c-f0e1-4486-882a-bf9275296975/konnectivity-agent/0.log" Jul 16 17:34:17.491097 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:17.491068 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-50.ec2.internal_683869ec7e8dd47873bd79d88a644cf6/haproxy/0.log" Jul 16 17:34:20.857253 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:20.857219 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75998hh8_4ed08f90-c844-43f1-9c65-ab8bede0ee73/extract/0.log" Jul 16 17:34:20.935308 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:20.935156 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75998hh8_4ed08f90-c844-43f1-9c65-ab8bede0ee73/util/0.log" Jul 16 17:34:21.047618 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:21.047581 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75998hh8_4ed08f90-c844-43f1-9c65-ab8bede0ee73/pull/0.log" Jul 16 17:34:21.171025 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:21.170930 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09psp4_04adff86-9c0b-495e-b73c-29da5b579fda/extract/0.log" Jul 16 17:34:21.272356 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:21.272322 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09psp4_04adff86-9c0b-495e-b73c-29da5b579fda/util/0.log" Jul 16 17:34:21.333277 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:21.333242 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09psp4_04adff86-9c0b-495e-b73c-29da5b579fda/pull/0.log" Jul 16 17:34:21.404270 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:21.404234 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rk7h4_a9b19b46-70a5-46be-b697-dcd420eed74c/extract/0.log" Jul 16 17:34:21.470507 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:21.470474 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rk7h4_a9b19b46-70a5-46be-b697-dcd420eed74c/util/0.log" Jul 16 17:34:21.525323 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:21.525292 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rk7h4_a9b19b46-70a5-46be-b697-dcd420eed74c/pull/0.log" Jul 16 17:34:21.581102 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:21.581070 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pfl45_8df978d6-8d77-40cf-a714-af26edb260f8/extract/0.log" Jul 16 17:34:21.624951 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:21.624918 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pfl45_8df978d6-8d77-40cf-a714-af26edb260f8/util/0.log" Jul 16 17:34:21.654895 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:21.654829 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pfl45_8df978d6-8d77-40cf-a714-af26edb260f8/pull/0.log" Jul 16 17:34:22.182673 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:22.182585 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-657df94667-8qzkj_7332c6ef-7c0f-4f06-9e27-f526a138a28b/authorino/0.log" Jul 16 17:34:22.219138 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:22.219107 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-ck96v_14df11f0-5fc9-491e-8e4a-54867cb706b0/manager/0.log" Jul 16 17:34:22.471789 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:22.471682 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-42p8f_238dde27-77a3-45a2-9ba4-ccc72f0bf58b/manager/0.log" Jul 16 17:34:24.427315 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:24.427275 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-5tsmr_bd2d4507-465f-4002-8953-9e289841cdbb/cluster-monitoring-operator/0.log" Jul 16 17:34:24.746567 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:24.746481 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fmz26_fdbc551c-5bfd-4f74-8050-523205d01489/node-exporter/0.log" Jul 16 17:34:24.780157 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:24.780125 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fmz26_fdbc551c-5bfd-4f74-8050-523205d01489/kube-rbac-proxy/0.log" Jul 16 17:34:24.821080 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:24.821050 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fmz26_fdbc551c-5bfd-4f74-8050-523205d01489/init-textfile/0.log" Jul 16 17:34:27.338872 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:27.338836 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-p8fjr/perf-node-gather-daemonset-cwl5j"] Jul 16 17:34:27.342625 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:27.342602 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-p8fjr/perf-node-gather-daemonset-cwl5j" Jul 16 17:34:27.352972 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:27.352949 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-p8fjr/perf-node-gather-daemonset-cwl5j"] Jul 16 17:34:27.409225 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:27.409151 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11ab6ec5-b942-4f28-bc71-a7957205ef3c-lib-modules\") pod \"perf-node-gather-daemonset-cwl5j\" (UID: \"11ab6ec5-b942-4f28-bc71-a7957205ef3c\") " pod="openshift-must-gather-p8fjr/perf-node-gather-daemonset-cwl5j" Jul 16 17:34:27.409421 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:27.409244 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11ab6ec5-b942-4f28-bc71-a7957205ef3c-sys\") pod \"perf-node-gather-daemonset-cwl5j\" (UID: \"11ab6ec5-b942-4f28-bc71-a7957205ef3c\") " pod="openshift-must-gather-p8fjr/perf-node-gather-daemonset-cwl5j" Jul 16 17:34:27.409421 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:27.409279 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/11ab6ec5-b942-4f28-bc71-a7957205ef3c-proc\") pod \"perf-node-gather-daemonset-cwl5j\" (UID: \"11ab6ec5-b942-4f28-bc71-a7957205ef3c\") " pod="openshift-must-gather-p8fjr/perf-node-gather-daemonset-cwl5j" Jul 16 17:34:27.409421 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:27.409312 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-948qw\" (UniqueName: \"kubernetes.io/projected/11ab6ec5-b942-4f28-bc71-a7957205ef3c-kube-api-access-948qw\") pod \"perf-node-gather-daemonset-cwl5j\" (UID: \"11ab6ec5-b942-4f28-bc71-a7957205ef3c\") " pod="openshift-must-gather-p8fjr/perf-node-gather-daemonset-cwl5j" Jul 16 17:34:27.409421 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:27.409361 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/11ab6ec5-b942-4f28-bc71-a7957205ef3c-podres\") pod \"perf-node-gather-daemonset-cwl5j\" (UID: \"11ab6ec5-b942-4f28-bc71-a7957205ef3c\") " pod="openshift-must-gather-p8fjr/perf-node-gather-daemonset-cwl5j" Jul 16 17:34:27.510920 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:27.510874 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/11ab6ec5-b942-4f28-bc71-a7957205ef3c-proc\") pod \"perf-node-gather-daemonset-cwl5j\" (UID: \"11ab6ec5-b942-4f28-bc71-a7957205ef3c\") " pod="openshift-must-gather-p8fjr/perf-node-gather-daemonset-cwl5j" Jul 16 17:34:27.511118 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:27.510949 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-948qw\" (UniqueName: \"kubernetes.io/projected/11ab6ec5-b942-4f28-bc71-a7957205ef3c-kube-api-access-948qw\") pod \"perf-node-gather-daemonset-cwl5j\" (UID: \"11ab6ec5-b942-4f28-bc71-a7957205ef3c\") " pod="openshift-must-gather-p8fjr/perf-node-gather-daemonset-cwl5j" Jul 16 17:34:27.511118 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:27.511003 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/11ab6ec5-b942-4f28-bc71-a7957205ef3c-podres\") pod \"perf-node-gather-daemonset-cwl5j\" (UID: \"11ab6ec5-b942-4f28-bc71-a7957205ef3c\") " pod="openshift-must-gather-p8fjr/perf-node-gather-daemonset-cwl5j" Jul 16 17:34:27.511118 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:27.511105 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11ab6ec5-b942-4f28-bc71-a7957205ef3c-lib-modules\") pod \"perf-node-gather-daemonset-cwl5j\" (UID: \"11ab6ec5-b942-4f28-bc71-a7957205ef3c\") " pod="openshift-must-gather-p8fjr/perf-node-gather-daemonset-cwl5j" Jul 16 17:34:27.511311 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:27.511138 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11ab6ec5-b942-4f28-bc71-a7957205ef3c-sys\") pod \"perf-node-gather-daemonset-cwl5j\" (UID: \"11ab6ec5-b942-4f28-bc71-a7957205ef3c\") " pod="openshift-must-gather-p8fjr/perf-node-gather-daemonset-cwl5j" Jul 16 17:34:27.511311 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:27.511248 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11ab6ec5-b942-4f28-bc71-a7957205ef3c-sys\") pod \"perf-node-gather-daemonset-cwl5j\" (UID: \"11ab6ec5-b942-4f28-bc71-a7957205ef3c\") " pod="openshift-must-gather-p8fjr/perf-node-gather-daemonset-cwl5j" Jul 16 17:34:27.511311 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:27.511307 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/11ab6ec5-b942-4f28-bc71-a7957205ef3c-proc\") pod \"perf-node-gather-daemonset-cwl5j\" (UID: \"11ab6ec5-b942-4f28-bc71-a7957205ef3c\") " pod="openshift-must-gather-p8fjr/perf-node-gather-daemonset-cwl5j" Jul 16 17:34:27.511745 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:27.511699 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/11ab6ec5-b942-4f28-bc71-a7957205ef3c-podres\") pod \"perf-node-gather-daemonset-cwl5j\" (UID: \"11ab6ec5-b942-4f28-bc71-a7957205ef3c\") " pod="openshift-must-gather-p8fjr/perf-node-gather-daemonset-cwl5j" Jul 16 17:34:27.511840 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:27.511819 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11ab6ec5-b942-4f28-bc71-a7957205ef3c-lib-modules\") pod \"perf-node-gather-daemonset-cwl5j\" (UID: \"11ab6ec5-b942-4f28-bc71-a7957205ef3c\") " pod="openshift-must-gather-p8fjr/perf-node-gather-daemonset-cwl5j" Jul 16 17:34:27.523144 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:27.523117 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-948qw\" (UniqueName: \"kubernetes.io/projected/11ab6ec5-b942-4f28-bc71-a7957205ef3c-kube-api-access-948qw\") pod \"perf-node-gather-daemonset-cwl5j\" (UID: \"11ab6ec5-b942-4f28-bc71-a7957205ef3c\") " pod="openshift-must-gather-p8fjr/perf-node-gather-daemonset-cwl5j" Jul 16 17:34:27.655782 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:27.655686 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-p8fjr/perf-node-gather-daemonset-cwl5j" Jul 16 17:34:27.826385 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:27.826333 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r8cn4_f637d961-672d-4aa2-a49a-fd021c2d4df2/console-operator/2.log" Jul 16 17:34:27.840135 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:27.840099 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r8cn4_f637d961-672d-4aa2-a49a-fd021c2d4df2/console-operator/3.log" Jul 16 17:34:27.855499 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:27.855062 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-p8fjr/perf-node-gather-daemonset-cwl5j"] Jul 16 17:34:27.857519 ip-10-0-128-50 kubenswrapper[2571]: W0716 17:34:27.857486 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod11ab6ec5_b942_4f28_bc71_a7957205ef3c.slice/crio-04baf0d926d5a5059a4debed80b74b949b9e23cc75a11f80ea5b5fbf69142898 WatchSource:0}: Error finding container 04baf0d926d5a5059a4debed80b74b949b9e23cc75a11f80ea5b5fbf69142898: Status 404 returned error can't find the container with id 04baf0d926d5a5059a4debed80b74b949b9e23cc75a11f80ea5b5fbf69142898 Jul 16 17:34:28.232965 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:28.232920 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-p8fjr/perf-node-gather-daemonset-cwl5j" event={"ID":"11ab6ec5-b942-4f28-bc71-a7957205ef3c","Type":"ContainerStarted","Data":"19c9153503039e9c1aa24d8f69e5c1b1bdec53376172cc74bb7f21afd432ee04"} Jul 16 17:34:28.233160 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:28.232993 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-p8fjr/perf-node-gather-daemonset-cwl5j" event={"ID":"11ab6ec5-b942-4f28-bc71-a7957205ef3c","Type":"ContainerStarted","Data":"04baf0d926d5a5059a4debed80b74b949b9e23cc75a11f80ea5b5fbf69142898"} Jul 16 17:34:28.233332 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:28.233309 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-p8fjr/perf-node-gather-daemonset-cwl5j" Jul 16 17:34:28.255539 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:28.255496 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-p8fjr/perf-node-gather-daemonset-cwl5j" podStartSLOduration=1.255481934 podStartE2EDuration="1.255481934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 17:34:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:34:28.254447281 +0000 UTC m=+2619.996022579" watchObservedRunningTime="2026-07-16 17:34:28.255481934 +0000 UTC m=+2619.997057230" Jul 16 17:34:28.389949 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:28.389919 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5b6cbd7f5f-f2mxc_4178dcc7-bf24-473e-b3f3-05e77c83e3a2/console/0.log" Jul 16 17:34:29.152475 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:29.152442 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-ct89n_1896b4d7-17ae-46be-98bc-54a304f42eda/volume-data-source-validator/0.log" Jul 16 17:34:30.310917 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:30.310880 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2bnf8_959268a1-18d8-4f7b-8c63-ba175ba63fe9/dns/0.log" Jul 16 17:34:30.344979 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:30.344948 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2bnf8_959268a1-18d8-4f7b-8c63-ba175ba63fe9/kube-rbac-proxy/0.log" Jul 16 17:34:30.527121 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:30.527089 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fttnf_f7e16013-94ca-4a61-b551-f3f3b66d3cb1/dns-node-resolver/0.log" Jul 16 17:34:31.263189 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:31.263131 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ln8q5_1dd02f2a-342f-46a3-bbba-b630378fa3db/node-ca/0.log" Jul 16 17:34:32.724575 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:32.724533 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-57bc6f48c6-kt2wd_d2bdea19-f5df-4297-8304-da19a8cc7a62/router/0.log" Jul 16 17:34:33.533969 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:33.533929 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xxcr4_d8306746-2174-40cd-9319-7137be8a4f5e/serve-healthcheck-canary/0.log" Jul 16 17:34:34.052327 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:34.052294 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-5t9rp_37bad57e-950d-4a6a-984b-19f240be539b/insights-operator/0.log" Jul 16 17:34:34.052802 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:34.052462 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-5t9rp_37bad57e-950d-4a6a-984b-19f240be539b/insights-operator/1.log" Jul 16 17:34:34.181990 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:34.181961 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cghnz_2c867807-c284-400a-a28c-b702c95abd6d/kube-rbac-proxy/0.log" Jul 16 17:34:34.208521 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:34.208488 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cghnz_2c867807-c284-400a-a28c-b702c95abd6d/exporter/0.log" Jul 16 17:34:34.243693 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:34.243665 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cghnz_2c867807-c284-400a-a28c-b702c95abd6d/extractor/0.log" Jul 16 17:34:34.251078 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:34.251051 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-p8fjr/perf-node-gather-daemonset-cwl5j" Jul 16 17:34:37.036672 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:37.036638 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-7zf2x_b162a8ad-e02f-4bb7-969a-b27787efcf7b/manager/0.log" Jul 16 17:34:37.278159 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:37.278124 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-85csp_28df4feb-0098-4de5-82c3-d983879f0f41/manager/0.log" Jul 16 17:34:37.458088 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:37.458060 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-57bc6f48c6-kt2wd_d2bdea19-f5df-4297-8304-da19a8cc7a62/router/0.log" Jul 16 17:34:37.467943 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:37.467920 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-l5d8m_581fb7d4-c35a-47cd-9d2f-78dce3928a1a/server/0.log" Jul 16 17:34:37.635622 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:37.635580 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c7f465c76-fwvtz_d4b8d3c4-3c0c-4fb9-9ad0-a3af9bd3495f/manager/0.log" Jul 16 17:34:37.661984 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:37.661963 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c7f465c76-fwvtz_d4b8d3c4-3c0c-4fb9-9ad0-a3af9bd3495f/copy-manifests/0.log" Jul 16 17:34:38.925990 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:38.925958 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-57bc6f48c6-kt2wd_d2bdea19-f5df-4297-8304-da19a8cc7a62/router/0.log" Jul 16 17:34:39.213429 ip-10-0-128-50 kubenswrapper[2571]: I0716 17:34:39.213335 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7c59ccc8d7-mgk5b_e651a638-235f-4c0e-8867-2c1eeace9b9d/manager/0.log"