Sep 11 11:47:46.268330 ip-10-0-132-170 systemd[1]: Starting Kubernetes Kubelet... Sep 11 11:47:46.695467 ip-10-0-132-170 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 11 11:47:46.695467 ip-10-0-132-170 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 11 11:47:46.695467 ip-10-0-132-170 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 11 11:47:46.695467 ip-10-0-132-170 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 11 11:47:46.695467 ip-10-0-132-170 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 11 11:47:46.696554 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.696466 2566 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 11 11:47:46.701544 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701518 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 11 11:47:46.701544 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701538 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 11 11:47:46.701544 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701543 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 11 11:47:46.701544 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701546 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 11 11:47:46.701544 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701550 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 11 11:47:46.701544 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701553 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 11 11:47:46.701772 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701556 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 11 11:47:46.701772 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701559 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 11 11:47:46.701772 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701562 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 11 11:47:46.701772 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701565 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 11 11:47:46.701772 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701567 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 11 11:47:46.701772 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701570 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 11 11:47:46.701772 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701573 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 11 11:47:46.701772 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701575 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 11 11:47:46.701772 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701580 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 11 11:47:46.701772 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701584 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 11 11:47:46.701772 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701587 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 11 11:47:46.701772 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701590 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 11 11:47:46.701772 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701592 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 11 11:47:46.701772 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701595 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 11 11:47:46.701772 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701599 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 11 11:47:46.701772 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701602 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 11 11:47:46.701772 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701606 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 11 11:47:46.701772 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701608 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 11 11:47:46.702234 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701611 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 11 11:47:46.702234 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701614 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 11 11:47:46.702234 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701617 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 11 11:47:46.702234 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701619 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 11 11:47:46.702234 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701622 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 11 11:47:46.702234 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701625 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 11 11:47:46.702234 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701627 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 11 11:47:46.702234 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701630 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 11 11:47:46.702234 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701635 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 11 11:47:46.702234 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701638 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 11 11:47:46.702234 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701641 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 11 11:47:46.702234 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701643 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 11 11:47:46.702234 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701646 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 11 11:47:46.702234 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701648 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 11 11:47:46.702234 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701651 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 11 11:47:46.702234 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701654 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 11 11:47:46.702234 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701657 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 11 11:47:46.702234 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701659 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 11 11:47:46.702234 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701661 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 11 11:47:46.702234 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701664 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 11 11:47:46.702722 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701667 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 11 11:47:46.702722 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701669 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 11 11:47:46.702722 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701672 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 11 11:47:46.702722 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701676 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 11 11:47:46.702722 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701678 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 11 11:47:46.702722 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701681 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 11 11:47:46.702722 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701683 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 11 11:47:46.702722 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701686 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 11 11:47:46.702722 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701688 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 11 11:47:46.702722 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701691 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 11 11:47:46.702722 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701693 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 11 11:47:46.702722 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701696 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 11 11:47:46.702722 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701699 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 11 11:47:46.702722 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701702 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 11 11:47:46.702722 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701706 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 11 11:47:46.702722 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701709 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 11 11:47:46.702722 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701712 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 11 11:47:46.702722 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701714 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 11 11:47:46.702722 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701717 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 11 11:47:46.702722 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701719 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 11 11:47:46.703226 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701722 2566 feature_gate.go:328] unrecognized feature gate: Example Sep 11 11:47:46.703226 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701724 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 11 11:47:46.703226 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701727 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 11 11:47:46.703226 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701729 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 11 11:47:46.703226 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701733 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 11 11:47:46.703226 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701735 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 11 11:47:46.703226 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701738 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 11 11:47:46.703226 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701741 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 11 11:47:46.703226 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701744 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 11 11:47:46.703226 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701747 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 11 11:47:46.703226 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701750 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 11 11:47:46.703226 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701753 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 11 11:47:46.703226 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701757 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 11 11:47:46.703226 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701760 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 11 11:47:46.703226 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701762 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 11 11:47:46.703226 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701765 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 11 11:47:46.703226 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701768 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 11 11:47:46.703226 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701770 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 11 11:47:46.703226 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701773 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 11 11:47:46.703226 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701776 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 11 11:47:46.703745 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701779 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 11 11:47:46.703745 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701782 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 11 11:47:46.703745 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701785 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 11 11:47:46.703745 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701788 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 11 11:47:46.703745 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701791 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 11 11:47:46.703745 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701793 2566 feature_gate.go:328] unrecognized feature gate: Example2 Sep 11 11:47:46.703745 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701796 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 11 11:47:46.703745 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701799 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 11 11:47:46.703745 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701802 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 11 11:47:46.703745 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701805 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 11 11:47:46.703745 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701807 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 11 11:47:46.703745 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701809 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 11 11:47:46.703745 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701812 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 11 11:47:46.703745 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701814 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 11 11:47:46.703745 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.701817 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 11 11:47:46.703745 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702550 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 11 11:47:46.703745 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702558 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 11 11:47:46.703745 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702561 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 11 11:47:46.703745 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702565 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 11 11:47:46.703745 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702568 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 11 11:47:46.704324 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702571 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 11 11:47:46.704324 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702574 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 11 11:47:46.704324 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702577 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 11 11:47:46.704324 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702580 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 11 11:47:46.704324 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702583 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 11 11:47:46.704324 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702585 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 11 11:47:46.704324 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702588 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 11 11:47:46.704324 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702591 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 11 11:47:46.704324 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702593 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 11 11:47:46.704324 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702596 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 11 11:47:46.704324 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702598 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 11 11:47:46.704324 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702602 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 11 11:47:46.704324 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702605 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 11 11:47:46.704324 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702608 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 11 11:47:46.704324 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702610 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 11 11:47:46.704324 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702613 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 11 11:47:46.704324 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702616 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 11 11:47:46.704324 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702618 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 11 11:47:46.704324 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702621 2566 feature_gate.go:328] unrecognized feature gate: Example2 Sep 11 11:47:46.704324 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702623 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 11 11:47:46.704871 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702626 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 11 11:47:46.704871 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702628 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 11 11:47:46.704871 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702631 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 11 11:47:46.704871 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702634 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 11 11:47:46.704871 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702636 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 11 11:47:46.704871 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702639 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 11 11:47:46.704871 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702644 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 11 11:47:46.704871 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702648 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 11 11:47:46.704871 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702651 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 11 11:47:46.704871 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702653 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 11 11:47:46.704871 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702658 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 11 11:47:46.704871 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702661 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 11 11:47:46.704871 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702663 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 11 11:47:46.704871 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702667 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 11 11:47:46.704871 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702670 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 11 11:47:46.704871 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702673 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 11 11:47:46.704871 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702675 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 11 11:47:46.704871 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702678 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 11 11:47:46.704871 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702680 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 11 11:47:46.705357 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702683 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 11 11:47:46.705357 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702686 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 11 11:47:46.705357 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702688 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 11 11:47:46.705357 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702690 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 11 11:47:46.705357 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702693 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 11 11:47:46.705357 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702695 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 11 11:47:46.705357 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702700 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 11 11:47:46.705357 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702703 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 11 11:47:46.705357 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702706 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 11 11:47:46.705357 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702708 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 11 11:47:46.705357 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702711 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 11 11:47:46.705357 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702714 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 11 11:47:46.705357 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702717 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 11 11:47:46.705357 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702719 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 11 11:47:46.705357 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702722 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 11 11:47:46.705357 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702725 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 11 11:47:46.705357 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702728 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 11 11:47:46.705357 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702731 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 11 11:47:46.705357 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702734 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 11 11:47:46.705823 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702737 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 11 11:47:46.705823 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702740 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 11 11:47:46.705823 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702742 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 11 11:47:46.705823 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702745 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 11 11:47:46.705823 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702748 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 11 11:47:46.705823 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702751 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 11 11:47:46.705823 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702754 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 11 11:47:46.705823 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702756 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 11 11:47:46.705823 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702759 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 11 11:47:46.705823 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702761 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 11 11:47:46.705823 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702763 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 11 11:47:46.705823 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702767 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 11 11:47:46.705823 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702769 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 11 11:47:46.705823 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702772 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 11 11:47:46.705823 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702774 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 11 11:47:46.705823 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702777 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 11 11:47:46.705823 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702779 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 11 11:47:46.705823 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702782 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 11 11:47:46.705823 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702784 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 11 11:47:46.706300 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702787 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 11 11:47:46.706300 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702793 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 11 11:47:46.706300 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702795 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 11 11:47:46.706300 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702798 2566 feature_gate.go:328] unrecognized feature gate: Example Sep 11 11:47:46.706300 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702801 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 11 11:47:46.706300 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702803 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 11 11:47:46.706300 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702806 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 11 11:47:46.706300 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702808 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 11 11:47:46.706300 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702811 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 11 11:47:46.706300 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702814 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 11 11:47:46.706300 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702816 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 11 11:47:46.706300 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702818 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 11 11:47:46.706300 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702821 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 11 11:47:46.706300 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702823 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 11 11:47:46.706300 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702826 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 11 11:47:46.706300 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702829 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 11 11:47:46.706300 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.702831 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 11 11:47:46.706300 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703244 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 11 11:47:46.706300 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703249 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 11 11:47:46.706300 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703252 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 11 11:47:46.706781 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703256 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 11 11:47:46.706781 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703258 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 11 11:47:46.706781 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703261 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 11 11:47:46.706781 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703264 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 11 11:47:46.706781 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703267 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 11 11:47:46.706781 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703270 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 11 11:47:46.706781 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703273 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 11 11:47:46.706781 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703275 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 11 11:47:46.706781 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703279 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 11 11:47:46.706781 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703281 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 11 11:47:46.706781 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703284 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 11 11:47:46.706781 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703286 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 11 11:47:46.706781 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703289 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 11 11:47:46.706781 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703292 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 11 11:47:46.706781 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703295 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 11 11:47:46.706781 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703297 2566 feature_gate.go:328] unrecognized feature gate: Example2 Sep 11 11:47:46.706781 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703300 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 11 11:47:46.706781 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703302 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 11 11:47:46.706781 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703305 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 11 11:47:46.706781 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703307 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 11 11:47:46.707277 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703310 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 11 11:47:46.707277 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703312 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 11 11:47:46.707277 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703315 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 11 11:47:46.707277 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703317 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 11 11:47:46.707277 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703320 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 11 11:47:46.707277 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703322 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 11 11:47:46.707277 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703324 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 11 11:47:46.707277 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703327 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 11 11:47:46.707277 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703330 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 11 11:47:46.707277 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703333 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 11 11:47:46.707277 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703336 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 11 11:47:46.707277 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703339 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 11 11:47:46.707277 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703342 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 11 11:47:46.707277 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703344 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 11 11:47:46.707277 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703346 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 11 11:47:46.707277 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703349 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 11 11:47:46.707277 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703351 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 11 11:47:46.707277 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703354 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 11 11:47:46.707277 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703356 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 11 11:47:46.707277 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703359 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 11 11:47:46.707765 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703361 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 11 11:47:46.707765 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703364 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 11 11:47:46.707765 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703366 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 11 11:47:46.707765 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703369 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 11 11:47:46.707765 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703371 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 11 11:47:46.707765 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703374 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 11 11:47:46.707765 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703377 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 11 11:47:46.707765 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703380 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 11 11:47:46.707765 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703383 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 11 11:47:46.707765 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703385 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 11 11:47:46.707765 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703388 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 11 11:47:46.707765 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703390 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 11 11:47:46.707765 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703395 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 11 11:47:46.707765 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703398 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 11 11:47:46.707765 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703402 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 11 11:47:46.707765 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703405 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 11 11:47:46.707765 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703407 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 11 11:47:46.707765 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703411 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 11 11:47:46.707765 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703414 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 11 11:47:46.708242 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703419 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 11 11:47:46.708242 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703423 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 11 11:47:46.708242 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703427 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 11 11:47:46.708242 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703430 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 11 11:47:46.708242 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703432 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 11 11:47:46.708242 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703435 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 11 11:47:46.708242 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703438 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 11 11:47:46.708242 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703441 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 11 11:47:46.708242 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703443 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 11 11:47:46.708242 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703446 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 11 11:47:46.708242 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703449 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 11 11:47:46.708242 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703452 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 11 11:47:46.708242 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703454 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 11 11:47:46.708242 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703456 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 11 11:47:46.708242 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703459 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 11 11:47:46.708242 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703462 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 11 11:47:46.708242 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703464 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 11 11:47:46.708242 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703466 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 11 11:47:46.708242 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703469 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 11 11:47:46.708696 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703471 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 11 11:47:46.708696 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703474 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 11 11:47:46.708696 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703477 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 11 11:47:46.708696 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703479 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 11 11:47:46.708696 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703481 2566 feature_gate.go:328] unrecognized feature gate: Example Sep 11 11:47:46.708696 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703484 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 11 11:47:46.708696 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703486 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 11 11:47:46.708696 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703489 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 11 11:47:46.708696 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703491 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 11 11:47:46.708696 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703494 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 11 11:47:46.708696 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703496 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 11 11:47:46.708696 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703498 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 11 11:47:46.708696 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703501 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 11 11:47:46.708696 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703503 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 11 11:47:46.708696 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703508 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 11 11:47:46.708696 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703512 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 11 11:47:46.708696 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703515 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 11 11:47:46.708696 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.703518 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 11 11:47:46.708696 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703589 2566 flags.go:64] FLAG: --address="0.0.0.0" Sep 11 11:47:46.708696 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703629 2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 11 11:47:46.709198 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703636 2566 flags.go:64] FLAG: --anonymous-auth="true" Sep 11 11:47:46.709198 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703640 2566 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 11 11:47:46.709198 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703644 2566 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 11 11:47:46.709198 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703648 2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 11 11:47:46.709198 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703652 2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 11 11:47:46.709198 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703659 2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 11 11:47:46.709198 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703662 2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 11 11:47:46.709198 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703665 2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 11 11:47:46.709198 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703669 2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 11 11:47:46.709198 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703672 2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 11 11:47:46.709198 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703676 2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 11 11:47:46.709198 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703679 2566 flags.go:64] FLAG: --cgroup-root="" Sep 11 11:47:46.709198 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703681 2566 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 11 11:47:46.709198 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703685 2566 flags.go:64] FLAG: --client-ca-file="" Sep 11 11:47:46.709198 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703688 2566 flags.go:64] FLAG: --cloud-provider="external" Sep 11 11:47:46.709198 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703691 2566 flags.go:64] FLAG: --cluster-dns="[]" Sep 11 11:47:46.709198 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703695 2566 flags.go:64] FLAG: --cluster-domain="" Sep 11 11:47:46.709198 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703698 2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 11 11:47:46.709198 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703701 2566 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 11 11:47:46.709198 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703704 2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 11 11:47:46.709198 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703708 2566 flags.go:64] FLAG: --container-log-max-files="5" Sep 11 11:47:46.709198 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703711 2566 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 11 11:47:46.709198 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703718 2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 11 11:47:46.709783 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703721 2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 11 11:47:46.709783 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703724 2566 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 11 11:47:46.709783 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703727 2566 flags.go:64] FLAG: --contention-profiling="false" Sep 11 11:47:46.709783 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703732 2566 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 11 11:47:46.709783 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703735 2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 11 11:47:46.709783 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703738 2566 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 11 11:47:46.709783 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703741 2566 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 11 11:47:46.709783 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703745 2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 11 11:47:46.709783 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703749 2566 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 11 11:47:46.709783 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703752 2566 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 11 11:47:46.709783 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703755 2566 flags.go:64] FLAG: --enable-load-reader="false" Sep 11 11:47:46.709783 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703758 2566 flags.go:64] FLAG: --enable-server="true" Sep 11 11:47:46.709783 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703761 2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 11 11:47:46.709783 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703766 2566 flags.go:64] FLAG: --event-burst="100" Sep 11 11:47:46.709783 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703769 2566 flags.go:64] FLAG: --event-qps="50" Sep 11 11:47:46.709783 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703772 2566 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 11 11:47:46.709783 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703775 2566 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 11 11:47:46.709783 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703778 2566 flags.go:64] FLAG: --eviction-hard="" Sep 11 11:47:46.709783 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703782 2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 11 11:47:46.709783 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703785 2566 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 11 11:47:46.709783 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703788 2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 11 11:47:46.709783 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703791 2566 flags.go:64] FLAG: --eviction-soft="" Sep 11 11:47:46.709783 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703794 2566 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 11 11:47:46.709783 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703797 2566 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 11 11:47:46.709783 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703801 2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 11 11:47:46.710402 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703804 2566 flags.go:64] FLAG: --experimental-mounter-path="" Sep 11 11:47:46.710402 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703807 2566 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 11 11:47:46.710402 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703810 2566 flags.go:64] FLAG: --fail-swap-on="true" Sep 11 11:47:46.710402 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703812 2566 flags.go:64] FLAG: --feature-gates="" Sep 11 11:47:46.710402 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703816 2566 flags.go:64] FLAG: --file-check-frequency="20s" Sep 11 11:47:46.710402 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703819 2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 11 11:47:46.710402 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703823 2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 11 11:47:46.710402 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703827 2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 11 11:47:46.710402 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703830 2566 flags.go:64] FLAG: --healthz-port="10248" Sep 11 11:47:46.710402 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703833 2566 flags.go:64] FLAG: --help="false" Sep 11 11:47:46.710402 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703837 2566 flags.go:64] FLAG: --hostname-override="ip-10-0-132-170.ec2.internal" Sep 11 11:47:46.710402 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703840 2566 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 11 11:47:46.710402 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703843 2566 flags.go:64] FLAG: --http-check-frequency="20s" Sep 11 11:47:46.710402 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703846 2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 11 11:47:46.710402 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703849 2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 11 11:47:46.710402 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703853 2566 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 11 11:47:46.710402 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703856 2566 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 11 11:47:46.710402 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703859 2566 flags.go:64] FLAG: --image-service-endpoint="" Sep 11 11:47:46.710402 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703861 2566 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 11 11:47:46.710402 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703864 2566 flags.go:64] FLAG: --kube-api-burst="100" Sep 11 11:47:46.710402 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703867 2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 11 11:47:46.710402 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703870 2566 flags.go:64] FLAG: --kube-api-qps="50" Sep 11 11:47:46.710402 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703873 2566 flags.go:64] FLAG: --kube-reserved="" Sep 11 11:47:46.710402 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703876 2566 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 11 11:47:46.711217 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703879 2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 11 11:47:46.711217 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703882 2566 flags.go:64] FLAG: --kubelet-cgroups="" Sep 11 11:47:46.711217 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703885 2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 11 11:47:46.711217 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703888 2566 flags.go:64] FLAG: --lock-file="" Sep 11 11:47:46.711217 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703890 2566 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 11 11:47:46.711217 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703893 2566 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 11 11:47:46.711217 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703896 2566 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 11 11:47:46.711217 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703901 2566 flags.go:64] FLAG: --log-json-split-stream="false" Sep 11 11:47:46.711217 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703905 2566 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 11 11:47:46.711217 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703907 2566 flags.go:64] FLAG: --log-text-split-stream="false" Sep 11 11:47:46.711217 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703910 2566 flags.go:64] FLAG: --logging-format="text" Sep 11 11:47:46.711217 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703913 2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 11 11:47:46.711217 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703917 2566 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 11 11:47:46.711217 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703919 2566 flags.go:64] FLAG: --manifest-url="" Sep 11 11:47:46.711217 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703924 2566 flags.go:64] FLAG: --manifest-url-header="" Sep 11 11:47:46.711217 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703928 2566 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 11 11:47:46.711217 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703931 2566 flags.go:64] FLAG: --max-open-files="1000000" Sep 11 11:47:46.711217 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703936 2566 flags.go:64] FLAG: --max-pods="110" Sep 11 11:47:46.711217 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703941 2566 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 11 11:47:46.711217 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703944 2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 11 11:47:46.711217 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703946 2566 flags.go:64] FLAG: --memory-manager-policy="None" Sep 11 11:47:46.711217 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703950 2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 11 11:47:46.711217 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703953 2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 11 11:47:46.711217 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703956 2566 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 11 11:47:46.711217 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703959 2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 11 11:47:46.712296 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703967 2566 flags.go:64] FLAG: --node-status-max-images="50" Sep 11 11:47:46.712296 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703970 2566 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 11 11:47:46.712296 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703974 2566 flags.go:64] FLAG: --oom-score-adj="-999" Sep 11 11:47:46.712296 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703976 2566 flags.go:64] FLAG: --pod-cidr="" Sep 11 11:47:46.712296 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703980 2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10" Sep 11 11:47:46.712296 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703985 2566 flags.go:64] FLAG: --pod-manifest-path="" Sep 11 11:47:46.712296 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703988 2566 flags.go:64] FLAG: --pod-max-pids="-1" Sep 11 11:47:46.712296 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703991 2566 flags.go:64] FLAG: --pods-per-core="0" Sep 11 11:47:46.712296 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703994 2566 flags.go:64] FLAG: --port="10250" Sep 11 11:47:46.712296 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.703997 2566 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 11 11:47:46.712296 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704000 2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-006752018659c4cc9" Sep 11 11:47:46.712296 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704003 2566 flags.go:64] FLAG: --qos-reserved="" Sep 11 11:47:46.712296 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704006 2566 flags.go:64] FLAG: --read-only-port="10255" Sep 11 11:47:46.712296 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704009 2566 flags.go:64] FLAG: --register-node="true" Sep 11 11:47:46.712296 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704012 2566 flags.go:64] FLAG: --register-with-taints="" Sep 11 11:47:46.712296 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704016 2566 flags.go:64] FLAG: --registry-burst="10" Sep 11 11:47:46.712296 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704020 2566 flags.go:64] FLAG: --registry-qps="5" Sep 11 11:47:46.712296 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704024 2566 flags.go:64] FLAG: --reserved-cpus="" Sep 11 11:47:46.712296 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704027 2566 flags.go:64] FLAG: --reserved-memory="" Sep 11 11:47:46.712296 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704031 2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 11 11:47:46.712296 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704049 2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 11 11:47:46.712296 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704052 2566 flags.go:64] FLAG: --rotate-certificates="false" Sep 11 11:47:46.712296 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704058 2566 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 11 11:47:46.712296 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704061 2566 flags.go:64] FLAG: --runonce="false" Sep 11 11:47:46.712296 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704064 2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 11 11:47:46.713245 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704068 2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 11 11:47:46.713245 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704072 2566 flags.go:64] FLAG: --seccomp-default="false" Sep 11 11:47:46.713245 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704074 2566 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 11 11:47:46.713245 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704077 2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 11 11:47:46.713245 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704080 2566 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 11 11:47:46.713245 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704084 2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 11 11:47:46.713245 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704087 2566 flags.go:64] FLAG: --storage-driver-password="root" Sep 11 11:47:46.713245 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704090 2566 flags.go:64] FLAG: --storage-driver-secure="false" Sep 11 11:47:46.713245 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704093 2566 flags.go:64] FLAG: --storage-driver-table="stats" Sep 11 11:47:46.713245 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704095 2566 flags.go:64] FLAG: --storage-driver-user="root" Sep 11 11:47:46.713245 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704098 2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 11 11:47:46.713245 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704101 2566 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 11 11:47:46.713245 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704104 2566 flags.go:64] FLAG: --system-cgroups="" Sep 11 11:47:46.713245 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704107 2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 11 11:47:46.713245 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704112 2566 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 11 11:47:46.713245 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704115 2566 flags.go:64] FLAG: --tls-cert-file="" Sep 11 11:47:46.713245 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704118 2566 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 11 11:47:46.713245 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704123 2566 flags.go:64] FLAG: --tls-min-version="" Sep 11 11:47:46.713245 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704126 2566 flags.go:64] FLAG: --tls-private-key-file="" Sep 11 11:47:46.713245 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704129 2566 flags.go:64] FLAG: --topology-manager-policy="none" Sep 11 11:47:46.713245 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704132 2566 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 11 11:47:46.713245 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704135 2566 flags.go:64] FLAG: --topology-manager-scope="container" Sep 11 11:47:46.713245 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704138 2566 flags.go:64] FLAG: --v="2" Sep 11 11:47:46.713245 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704142 2566 flags.go:64] FLAG: --version="false" Sep 11 11:47:46.713245 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704146 2566 flags.go:64] FLAG: --vmodule="" Sep 11 11:47:46.713849 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704150 2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 11 11:47:46.713849 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704153 2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 11 11:47:46.713849 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704276 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 11 11:47:46.713849 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704279 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 11 11:47:46.713849 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704284 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 11 11:47:46.713849 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704288 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 11 11:47:46.713849 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704291 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 11 11:47:46.713849 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704295 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 11 11:47:46.713849 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704298 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 11 11:47:46.713849 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704301 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 11 11:47:46.713849 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704303 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 11 11:47:46.713849 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704306 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 11 11:47:46.713849 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704309 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 11 11:47:46.713849 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704312 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 11 11:47:46.713849 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704314 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 11 11:47:46.713849 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704317 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 11 11:47:46.713849 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704320 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 11 11:47:46.713849 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704322 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 11 11:47:46.713849 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704325 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 11 11:47:46.714704 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704328 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 11 11:47:46.714704 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704331 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 11 11:47:46.714704 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704334 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 11 11:47:46.714704 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704336 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 11 11:47:46.714704 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704339 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 11 11:47:46.714704 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704341 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 11 11:47:46.714704 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704344 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 11 11:47:46.714704 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704347 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 11 11:47:46.714704 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704349 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 11 11:47:46.714704 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704352 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 11 11:47:46.714704 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704354 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 11 11:47:46.714704 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704357 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 11 11:47:46.714704 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704360 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 11 11:47:46.714704 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704363 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 11 11:47:46.714704 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704365 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 11 11:47:46.714704 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704368 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 11 11:47:46.714704 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704371 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 11 11:47:46.714704 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704374 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 11 11:47:46.714704 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704377 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 11 11:47:46.714704 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704380 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 11 11:47:46.714704 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704383 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 11 11:47:46.715547 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704386 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 11 11:47:46.715547 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704388 2566 feature_gate.go:328] unrecognized feature gate: Example Sep 11 11:47:46.715547 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704391 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 11 11:47:46.715547 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704394 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 11 11:47:46.715547 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704396 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 11 11:47:46.715547 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704399 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 11 11:47:46.715547 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704402 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 11 11:47:46.715547 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704404 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 11 11:47:46.715547 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704407 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 11 11:47:46.715547 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704409 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 11 11:47:46.715547 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704414 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 11 11:47:46.715547 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704417 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 11 11:47:46.715547 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704420 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 11 11:47:46.715547 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704423 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 11 11:47:46.715547 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704426 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 11 11:47:46.715547 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704429 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 11 11:47:46.715547 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704431 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 11 11:47:46.715547 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704434 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 11 11:47:46.715547 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704436 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 11 11:47:46.715547 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704439 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 11 11:47:46.716118 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704442 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 11 11:47:46.716118 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704444 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 11 11:47:46.716118 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704447 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 11 11:47:46.716118 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704449 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 11 11:47:46.716118 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704452 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 11 11:47:46.716118 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704455 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 11 11:47:46.716118 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704459 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 11 11:47:46.716118 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704463 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 11 11:47:46.716118 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704469 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 11 11:47:46.716118 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704472 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 11 11:47:46.716118 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704476 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 11 11:47:46.716118 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704481 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 11 11:47:46.716118 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704485 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 11 11:47:46.716118 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704490 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 11 11:47:46.716118 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704495 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 11 11:47:46.716118 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704499 2566 feature_gate.go:328] unrecognized feature gate: Example2 Sep 11 11:47:46.716118 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704502 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 11 11:47:46.716118 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704505 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 11 11:47:46.716855 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704507 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 11 11:47:46.716855 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704510 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 11 11:47:46.716855 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704513 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 11 11:47:46.716855 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704515 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 11 11:47:46.716855 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704518 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 11 11:47:46.716855 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704521 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 11 11:47:46.716855 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704523 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 11 11:47:46.716855 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704526 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 11 11:47:46.716855 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704529 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 11 11:47:46.716855 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704532 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 11 11:47:46.716855 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704535 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 11 11:47:46.716855 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704537 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 11 11:47:46.716855 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704540 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 11 11:47:46.716855 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704542 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 11 11:47:46.716855 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704545 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 11 11:47:46.716855 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704548 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 11 11:47:46.716855 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704550 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 11 11:47:46.716855 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704553 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 11 11:47:46.716855 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704555 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 11 11:47:46.716855 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704558 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 11 11:47:46.717624 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704561 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 11 11:47:46.717624 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704564 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 11 11:47:46.717624 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.704568 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 11 11:47:46.717624 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.704576 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 11 11:47:46.717624 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.712985 2566 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 11 11:47:46.717624 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.713003 2566 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 11 11:47:46.717624 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.713901 2566 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 11 11:47:46.717624 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.713915 2566 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 11 11:47:46.717624 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714003 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 11 11:47:46.717624 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714012 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 11 11:47:46.717624 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714018 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 11 11:47:46.717624 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714025 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 11 11:47:46.717624 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714030 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 11 11:47:46.717624 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714050 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 11 11:47:46.717978 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714055 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 11 11:47:46.717978 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714060 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 11 11:47:46.717978 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714065 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 11 11:47:46.717978 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714069 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 11 11:47:46.717978 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714073 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 11 11:47:46.717978 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714077 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 11 11:47:46.717978 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714081 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 11 11:47:46.717978 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714086 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 11 11:47:46.717978 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714090 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 11 11:47:46.717978 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714095 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 11 11:47:46.717978 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714099 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 11 11:47:46.717978 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714103 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 11 11:47:46.717978 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714108 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 11 11:47:46.717978 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714112 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 11 11:47:46.717978 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714116 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 11 11:47:46.717978 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714120 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 11 11:47:46.717978 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714124 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 11 11:47:46.717978 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714128 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 11 11:47:46.717978 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714133 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 11 11:47:46.717978 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714137 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 11 11:47:46.718625 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714141 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 11 11:47:46.718625 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714146 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 11 11:47:46.718625 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714152 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 11 11:47:46.718625 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714157 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 11 11:47:46.718625 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714161 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 11 11:47:46.718625 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714166 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 11 11:47:46.718625 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714170 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 11 11:47:46.718625 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714174 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 11 11:47:46.718625 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714178 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 11 11:47:46.718625 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714185 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 11 11:47:46.718625 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714191 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 11 11:47:46.718625 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714195 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 11 11:47:46.718625 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714199 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 11 11:47:46.718625 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714204 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 11 11:47:46.718625 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714208 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 11 11:47:46.718625 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714212 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 11 11:47:46.718625 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714217 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 11 11:47:46.718625 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714221 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 11 11:47:46.718625 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714225 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 11 11:47:46.718625 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714229 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 11 11:47:46.719166 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714234 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 11 11:47:46.719166 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714238 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 11 11:47:46.719166 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714243 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 11 11:47:46.719166 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714247 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 11 11:47:46.719166 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714252 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 11 11:47:46.719166 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714258 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 11 11:47:46.719166 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714263 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 11 11:47:46.719166 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714267 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 11 11:47:46.719166 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714271 2566 feature_gate.go:328] unrecognized feature gate: Example2 Sep 11 11:47:46.719166 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714276 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 11 11:47:46.719166 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714280 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 11 11:47:46.719166 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714284 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 11 11:47:46.719166 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714288 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 11 11:47:46.719166 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714295 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 11 11:47:46.719166 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714301 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 11 11:47:46.719166 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714307 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 11 11:47:46.719166 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714316 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 11 11:47:46.719166 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714321 2566 feature_gate.go:328] unrecognized feature gate: Example Sep 11 11:47:46.719166 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714326 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 11 11:47:46.719166 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714330 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 11 11:47:46.719702 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714334 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 11 11:47:46.719702 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714339 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 11 11:47:46.719702 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714343 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 11 11:47:46.719702 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714348 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 11 11:47:46.719702 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714353 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 11 11:47:46.719702 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714358 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 11 11:47:46.719702 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714363 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 11 11:47:46.719702 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714368 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 11 11:47:46.719702 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714372 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 11 11:47:46.719702 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714376 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 11 11:47:46.719702 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714381 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 11 11:47:46.719702 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714385 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 11 11:47:46.719702 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714389 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 11 11:47:46.719702 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714394 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 11 11:47:46.719702 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714399 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 11 11:47:46.719702 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714403 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 11 11:47:46.719702 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714407 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 11 11:47:46.719702 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714411 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 11 11:47:46.719702 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714416 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 11 11:47:46.719702 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714420 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 11 11:47:46.720379 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714425 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 11 11:47:46.720379 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714429 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 11 11:47:46.720379 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714433 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 11 11:47:46.720379 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714437 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 11 11:47:46.720379 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714441 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 11 11:47:46.720379 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714446 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 11 11:47:46.720379 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714450 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 11 11:47:46.720379 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714454 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 11 11:47:46.720379 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714458 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 11 11:47:46.720379 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714463 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 11 11:47:46.720379 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714468 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 11 11:47:46.720379 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714472 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 11 11:47:46.720379 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714476 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 11 11:47:46.720379 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.714485 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 11 11:47:46.720731 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714670 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 11 11:47:46.720731 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714679 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 11 11:47:46.720731 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714684 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 11 11:47:46.720731 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714689 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 11 11:47:46.720731 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714693 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 11 11:47:46.720731 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714697 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 11 11:47:46.720731 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714701 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 11 11:47:46.720731 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714705 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 11 11:47:46.720731 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714709 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 11 11:47:46.720731 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714713 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 11 11:47:46.720731 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714717 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 11 11:47:46.720731 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714722 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 11 11:47:46.720731 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714727 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 11 11:47:46.720731 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714731 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 11 11:47:46.720731 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714735 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 11 11:47:46.720731 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714740 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 11 11:47:46.720731 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714744 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 11 11:47:46.720731 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714748 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 11 11:47:46.720731 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714755 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 11 11:47:46.720731 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714761 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 11 11:47:46.721271 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714766 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 11 11:47:46.721271 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714770 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 11 11:47:46.721271 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714775 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 11 11:47:46.721271 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714779 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 11 11:47:46.721271 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714784 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 11 11:47:46.721271 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714788 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 11 11:47:46.721271 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714793 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 11 11:47:46.721271 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714797 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 11 11:47:46.721271 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714802 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 11 11:47:46.721271 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714806 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 11 11:47:46.721271 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714811 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 11 11:47:46.721271 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714815 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 11 11:47:46.721271 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714819 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 11 11:47:46.721271 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714822 2566 feature_gate.go:328] unrecognized feature gate: Example Sep 11 11:47:46.721271 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714827 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 11 11:47:46.721271 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714830 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 11 11:47:46.721271 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714835 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 11 11:47:46.721271 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714839 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 11 11:47:46.721271 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714843 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 11 11:47:46.721743 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714848 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 11 11:47:46.721743 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714852 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 11 11:47:46.721743 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714856 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 11 11:47:46.721743 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714861 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 11 11:47:46.721743 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714865 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 11 11:47:46.721743 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714869 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 11 11:47:46.721743 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714873 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 11 11:47:46.721743 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714877 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 11 11:47:46.721743 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714881 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 11 11:47:46.721743 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714885 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 11 11:47:46.721743 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714889 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 11 11:47:46.721743 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714894 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 11 11:47:46.721743 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714898 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 11 11:47:46.721743 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714902 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 11 11:47:46.721743 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714906 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 11 11:47:46.721743 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714910 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 11 11:47:46.721743 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714914 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 11 11:47:46.721743 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714919 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 11 11:47:46.721743 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714923 2566 feature_gate.go:328] unrecognized feature gate: Example2 Sep 11 11:47:46.721743 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714928 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 11 11:47:46.722447 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714932 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 11 11:47:46.722447 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714938 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 11 11:47:46.722447 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714944 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 11 11:47:46.722447 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714950 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 11 11:47:46.722447 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714956 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 11 11:47:46.722447 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714961 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 11 11:47:46.722447 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714965 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 11 11:47:46.722447 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714969 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 11 11:47:46.722447 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714973 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 11 11:47:46.722447 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714977 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 11 11:47:46.722447 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714981 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 11 11:47:46.722447 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714986 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 11 11:47:46.722447 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714990 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 11 11:47:46.722447 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714995 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 11 11:47:46.722447 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.714999 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 11 11:47:46.722447 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.715003 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 11 11:47:46.722447 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.715008 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 11 11:47:46.722447 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.715012 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 11 11:47:46.722447 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.715016 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 11 11:47:46.722921 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.715020 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 11 11:47:46.722921 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.715024 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 11 11:47:46.722921 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.715028 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 11 11:47:46.722921 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.715049 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 11 11:47:46.722921 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.715053 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 11 11:47:46.722921 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.715057 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 11 11:47:46.722921 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.715062 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 11 11:47:46.722921 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.715066 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 11 11:47:46.722921 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.715070 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 11 11:47:46.722921 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.715074 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 11 11:47:46.722921 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.715079 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 11 11:47:46.722921 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.715083 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 11 11:47:46.722921 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.715088 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 11 11:47:46.722921 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.715092 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 11 11:47:46.722921 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.715096 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 11 11:47:46.722921 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.715100 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 11 11:47:46.722921 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.715105 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 11 11:47:46.722921 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.715110 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 11 11:47:46.722921 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.715113 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 11 11:47:46.722921 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.715117 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 11 11:47:46.723566 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:46.715121 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 11 11:47:46.723566 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.715129 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 11 11:47:46.723566 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.715276 2566 server.go:962] "Client rotation is on, will bootstrap in background" Sep 11 11:47:46.723566 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.718172 2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 11 11:47:46.723566 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.719259 2566 server.go:1019] "Starting client certificate rotation" Sep 11 11:47:46.723566 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.719364 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 11 11:47:46.723566 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.719403 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 11 11:47:46.723566 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.723483 2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 11 11:47:46.732689 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.732667 2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 11 11:47:46.742367 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.742345 2566 log.go:25] "Validated CRI v1 runtime API" Sep 11 11:47:46.745413 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.745399 2566 log.go:25] "Validated CRI v1 image API" Sep 11 11:47:46.747708 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.747695 2566 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 11 11:47:46.751534 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.751515 2566 fs.go:135] Filesystem UUIDs: map[53c525ee-ead4-45bd-843e-68b6b2c42a36:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 87f713b3-4399-4217-ac52-ff54856669ea:/dev/nvme0n1p3] Sep 11 11:47:46.751580 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.751536 2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 11 11:47:46.756633 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.756527 2566 manager.go:216] Machine: {Timestamp:2026-09-11 11:47:46.755226628 +0000 UTC m=+0.376029603 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3109672 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20e0c7222d6f307b79f82b454a2fd4 SystemUUID:ec20e0c7-222d-6f30-7b79-f82b454a2fd4 BootID:d9dac31d-0cac-4119-b0f1-db52f517b835 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:df:a0:74:ff:db Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:df:a0:74:ff:db Speed:0 Mtu:9001} {Name:ovs-system MacAddress:42:e1:56:ae:36:f7 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 11 11:47:46.756633 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.756629 2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 11 11:47:46.756735 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.756713 2566 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 11 11:47:46.759560 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.759543 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 11 11:47:46.760297 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.760271 2566 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 11 11:47:46.760429 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.760302 2566 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-170.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 11 11:47:46.761786 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.761775 2566 topology_manager.go:138] "Creating topology manager with none policy" Sep 11 11:47:46.761822 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.761788 2566 container_manager_linux.go:309] "Creating device plugin manager" Sep 11 11:47:46.761822 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.761801 2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 11 11:47:46.762586 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.762576 2566 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 11 11:47:46.763559 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.763549 2566 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 11 11:47:46.767664 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.767652 2566 state_mem.go:36] "Initialized new in-memory state store" Sep 11 11:47:46.767815 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.767806 2566 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 11 11:47:46.767850 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.767843 2566 kubelet.go:486] "Attempting to sync node with API server" Sep 11 11:47:46.767880 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.767853 2566 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 11 11:47:46.767880 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.767864 2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 11 11:47:46.767880 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.767873 2566 kubelet.go:398] "Adding apiserver pod source" Sep 11 11:47:46.767958 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.767881 2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 11 11:47:46.768502 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.768483 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 11 11:47:46.768604 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.768517 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 11 11:47:46.770934 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.770912 2566 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 11 11:47:46.774017 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.774001 2566 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 11 11:47:46.774101 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.774047 2566 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 11 11:47:46.775783 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.775770 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 11 11:47:46.775831 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.775787 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 11 11:47:46.775831 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.775793 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 11 11:47:46.775831 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.775799 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 11 11:47:46.775831 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.775807 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 11 11:47:46.775831 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.775813 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 11 11:47:46.775971 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.775836 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 11 11:47:46.775971 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.775842 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 11 11:47:46.775971 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.775850 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 11 11:47:46.775971 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.775856 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 11 11:47:46.775971 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.775865 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 11 11:47:46.775971 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.775870 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 11 11:47:46.777619 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.777606 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 11 11:47:46.777619 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.777616 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 11 11:47:46.779246 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:46.779225 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-170.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 11 11:47:46.779246 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:46.779207 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 11 11:47:46.781443 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.781429 2566 server.go:1268] "Started kubelet" Sep 11 11:47:46.785364 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.785329 2566 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 11 11:47:46.785437 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.785405 2566 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 11 11:47:46.785485 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.785407 2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 11 11:47:46.786207 ip-10-0-132-170 systemd[1]: Started Kubernetes Kubelet. Sep 11 11:47:46.787094 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.786785 2566 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 11 11:47:46.788255 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.788239 2566 server.go:310] "Adding debug handlers to kubelet server" Sep 11 11:47:46.790773 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:46.790755 2566 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 11 11:47:46.793249 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.793232 2566 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-170.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 11 11:47:46.793763 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:46.792711 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-170.ec2.internal.18d441c51a7887f2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-170.ec2.internal,UID:ip-10-0-132-170.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-170.ec2.internal,},FirstTimestamp:2026-09-11 11:47:46.781399026 +0000 UTC m=+0.402202001,LastTimestamp:2026-09-11 11:47:46.781399026 +0000 UTC m=+0.402202001,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-170.ec2.internal,}" Sep 11 11:47:46.795815 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.795797 2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 11 11:47:46.795815 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.795812 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 11 11:47:46.795959 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:46.795936 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-170.ec2.internal\" not found" Sep 11 11:47:46.796498 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.796477 2566 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 11 11:47:46.796498 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.796478 2566 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 11 11:47:46.796623 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.796507 2566 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 11 11:47:46.796623 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.796606 2566 reconstruct.go:97] "Volume reconstruction finished" Sep 11 11:47:46.796623 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.796615 2566 reconciler.go:29] "Reconciler: start to sync state" Sep 11 11:47:46.796812 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.796789 2566 factory.go:55] Registering systemd factory Sep 11 11:47:46.796871 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.796815 2566 factory.go:223] Registration of the systemd container factory successfully Sep 11 11:47:46.797020 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.797006 2566 factory.go:153] Registering CRI-O factory Sep 11 11:47:46.797020 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.797021 2566 factory.go:223] Registration of the crio container factory successfully Sep 11 11:47:46.797123 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.797079 2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 11 11:47:46.797123 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.797094 2566 factory.go:103] Registering Raw factory Sep 11 11:47:46.797123 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.797106 2566 manager.go:1196] Started watching for new ooms in manager Sep 11 11:47:46.797473 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.797458 2566 manager.go:349] Starting recovery of all containers Sep 11 11:47:46.798985 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:46.798954 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 11 11:47:46.798985 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:46.798954 2566 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-170.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 11 11:47:46.804799 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.804781 2566 manager.go:354] Recovery completed Sep 11 11:47:46.809334 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.809220 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 11 11:47:46.810977 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.810960 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-170.ec2.internal" event="NodeHasSufficientMemory" Sep 11 11:47:46.811061 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.811002 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-170.ec2.internal" event="NodeHasNoDiskPressure" Sep 11 11:47:46.811061 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.811014 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-170.ec2.internal" event="NodeHasSufficientPID" Sep 11 11:47:46.811469 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.811457 2566 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 11 11:47:46.811469 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.811468 2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 11 11:47:46.811573 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.811484 2566 state_mem.go:36] "Initialized new in-memory state store" Sep 11 11:47:46.811573 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.811529 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 11 11:47:46.813362 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:46.813297 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-170.ec2.internal.18d441c51c3bd7fa default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-170.ec2.internal,UID:ip-10-0-132-170.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-170.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-170.ec2.internal,},FirstTimestamp:2026-09-11 11:47:46.81097625 +0000 UTC m=+0.431779226,LastTimestamp:2026-09-11 11:47:46.81097625 +0000 UTC m=+0.431779226,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-170.ec2.internal,}" Sep 11 11:47:46.813655 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.813638 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-170.ec2.internal" event="NodeHasSufficientMemory" Sep 11 11:47:46.813733 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.813673 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-170.ec2.internal" event="NodeHasNoDiskPressure" Sep 11 11:47:46.813733 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.813689 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-170.ec2.internal" event="NodeHasSufficientPID" Sep 11 11:47:46.813733 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.813703 2566 policy_none.go:49] "None policy: Start" Sep 11 11:47:46.813733 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.813721 2566 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 11 11:47:46.813915 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.813736 2566 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 11 11:47:46.815778 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.815765 2566 policy_none.go:47] "Start" Sep 11 11:47:46.820538 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.820520 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bt44g" Sep 11 11:47:46.824537 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:46.824478 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-170.ec2.internal.18d441c51c3c553e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-170.ec2.internal,UID:ip-10-0-132-170.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-132-170.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-132-170.ec2.internal,},FirstTimestamp:2026-09-11 11:47:46.811008318 +0000 UTC m=+0.431811292,LastTimestamp:2026-09-11 11:47:46.811008318 +0000 UTC m=+0.431811292,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-170.ec2.internal,}" Sep 11 11:47:46.828546 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.828525 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bt44g" Sep 11 11:47:46.834295 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:46.834223 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-170.ec2.internal.18d441c51c3c7b36 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-170.ec2.internal,UID:ip-10-0-132-170.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-132-170.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-132-170.ec2.internal,},FirstTimestamp:2026-09-11 11:47:46.811018038 +0000 UTC m=+0.431821013,LastTimestamp:2026-09-11 11:47:46.811018038 +0000 UTC m=+0.431821013,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-170.ec2.internal,}" Sep 11 11:47:46.846613 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.846598 2566 manager.go:337] "Starting Device Plugin manager" Sep 11 11:47:46.874215 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:46.846644 2566 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 11 11:47:46.874215 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.846656 2566 server.go:87] "Starting device plugin registration server" Sep 11 11:47:46.874215 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.846954 2566 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 11 11:47:46.874215 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.846968 2566 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 11 11:47:46.874215 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.847139 2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 11 11:47:46.874215 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.847229 2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 11 11:47:46.874215 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.847238 2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 11 11:47:46.874215 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:46.847676 2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 11 11:47:46.874215 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:46.847726 2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-170.ec2.internal\" not found" Sep 11 11:47:46.894858 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.894833 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 11 11:47:46.896079 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.896062 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 11 11:47:46.896141 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.896117 2566 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 11 11:47:46.896186 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.896146 2566 kubelet.go:2448] "Starting kubelet main sync loop" Sep 11 11:47:46.896260 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:46.896243 2566 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 11 11:47:46.898532 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.898512 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 11 11:47:46.947770 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.947668 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 11 11:47:46.948607 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.948586 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-170.ec2.internal" event="NodeHasSufficientMemory" Sep 11 11:47:46.948713 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.948624 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-170.ec2.internal" event="NodeHasNoDiskPressure" Sep 11 11:47:46.948713 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.948638 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-170.ec2.internal" event="NodeHasSufficientPID" Sep 11 11:47:46.948713 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.948670 2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-170.ec2.internal" Sep 11 11:47:46.969284 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.969266 2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-170.ec2.internal" Sep 11 11:47:46.969365 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:46.969290 2566 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-170.ec2.internal\": node \"ip-10-0-132-170.ec2.internal\" not found" Sep 11 11:47:46.991330 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:46.991303 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-170.ec2.internal\" not found" Sep 11 11:47:46.996389 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.996371 2566 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-132-170.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-170.ec2.internal"] Sep 11 11:47:46.996476 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.996441 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 11 11:47:46.996899 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.996879 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/579298bbec0b9e8e74fae2628d563376-config\") pod \"kube-apiserver-proxy-ip-10-0-132-170.ec2.internal\" (UID: \"579298bbec0b9e8e74fae2628d563376\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-170.ec2.internal" Sep 11 11:47:46.999222 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.999206 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-170.ec2.internal" event="NodeHasSufficientMemory" Sep 11 11:47:46.999269 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.999237 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-170.ec2.internal" event="NodeHasNoDiskPressure" Sep 11 11:47:46.999269 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:46.999248 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-170.ec2.internal" event="NodeHasSufficientPID" Sep 11 11:47:47.001500 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:47.001489 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 11 11:47:47.001630 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:47.001614 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-170.ec2.internal" Sep 11 11:47:47.001675 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:47.001647 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 11 11:47:47.002232 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:47.002219 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-170.ec2.internal" event="NodeHasSufficientMemory" Sep 11 11:47:47.002297 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:47.002239 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-170.ec2.internal" event="NodeHasSufficientMemory" Sep 11 11:47:47.002297 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:47.002244 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-170.ec2.internal" event="NodeHasNoDiskPressure" Sep 11 11:47:47.002297 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:47.002254 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-170.ec2.internal" event="NodeHasNoDiskPressure" Sep 11 11:47:47.002297 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:47.002255 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-170.ec2.internal" event="NodeHasSufficientPID" Sep 11 11:47:47.002297 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:47.002266 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-170.ec2.internal" event="NodeHasSufficientPID" Sep 11 11:47:47.004424 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:47.004412 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-170.ec2.internal" Sep 11 11:47:47.004473 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:47.004437 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 11 11:47:47.005086 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:47.005072 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-170.ec2.internal" event="NodeHasSufficientMemory" Sep 11 11:47:47.005170 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:47.005099 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-170.ec2.internal" event="NodeHasNoDiskPressure" Sep 11 11:47:47.005170 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:47.005112 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-170.ec2.internal" event="NodeHasSufficientPID" Sep 11 11:47:47.019585 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:47.019566 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-170.ec2.internal\" not found" node="ip-10-0-132-170.ec2.internal" Sep 11 11:47:47.024199 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:47.024183 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-170.ec2.internal\" not found" node="ip-10-0-132-170.ec2.internal" Sep 11 11:47:47.091428 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:47.091398 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-170.ec2.internal\" not found" Sep 11 11:47:47.097917 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:47.097900 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/579298bbec0b9e8e74fae2628d563376-config\") pod \"kube-apiserver-proxy-ip-10-0-132-170.ec2.internal\" (UID: \"579298bbec0b9e8e74fae2628d563376\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-170.ec2.internal" Sep 11 11:47:47.097980 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:47.097925 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bf7554308d9b300ebecd820a92d9c922-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-170.ec2.internal\" (UID: \"bf7554308d9b300ebecd820a92d9c922\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-170.ec2.internal" Sep 11 11:47:47.097980 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:47.097943 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bf7554308d9b300ebecd820a92d9c922-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-170.ec2.internal\" (UID: \"bf7554308d9b300ebecd820a92d9c922\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-170.ec2.internal" Sep 11 11:47:47.098092 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:47.098016 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/579298bbec0b9e8e74fae2628d563376-config\") pod \"kube-apiserver-proxy-ip-10-0-132-170.ec2.internal\" (UID: \"579298bbec0b9e8e74fae2628d563376\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-170.ec2.internal" Sep 11 11:47:47.191917 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:47.191889 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-170.ec2.internal\" not found" Sep 11 11:47:47.198895 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:47.198841 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bf7554308d9b300ebecd820a92d9c922-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-170.ec2.internal\" (UID: \"bf7554308d9b300ebecd820a92d9c922\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-170.ec2.internal" Sep 11 11:47:47.198895 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:47.198870 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bf7554308d9b300ebecd820a92d9c922-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-170.ec2.internal\" (UID: \"bf7554308d9b300ebecd820a92d9c922\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-170.ec2.internal" Sep 11 11:47:47.199029 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:47.198914 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bf7554308d9b300ebecd820a92d9c922-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-170.ec2.internal\" (UID: \"bf7554308d9b300ebecd820a92d9c922\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-170.ec2.internal" Sep 11 11:47:47.199029 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:47.198959 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bf7554308d9b300ebecd820a92d9c922-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-170.ec2.internal\" (UID: \"bf7554308d9b300ebecd820a92d9c922\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-170.ec2.internal" Sep 11 11:47:47.292232 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:47.292198 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-170.ec2.internal\" not found" Sep 11 11:47:47.322744 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:47.322724 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-170.ec2.internal" Sep 11 11:47:47.326430 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:47.326408 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-170.ec2.internal" Sep 11 11:47:47.392301 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:47.392258 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-170.ec2.internal\" not found" Sep 11 11:47:49.348694 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:49.348629 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-170.ec2.internal\" not found" Sep 11 11:47:49.449114 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:49.449067 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-170.ec2.internal\" not found" Sep 11 11:47:49.549680 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:49.549633 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-170.ec2.internal\" not found" Sep 11 11:47:49.575069 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:49.575044 2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 11 11:47:49.575487 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:49.575214 2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 11 11:47:49.639261 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:49.639198 2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 11 11:47:49.652727 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:49.652706 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 11 11:47:49.652727 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:49.652715 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-170.ec2.internal" Sep 11 11:47:49.668771 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:49.668751 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 11 11:47:49.670418 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:49.670404 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-170.ec2.internal" Sep 11 11:47:49.670974 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:49.670959 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 11 11:47:49.684924 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:49.684907 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 11 11:47:49.695748 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:49.695727 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-10 11:42:48 +0000 UTC" deadline="2028-04-06 22:26:33.760785584 +0000 UTC" Sep 11 11:47:49.695791 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:49.695748 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13762h38m44.065039661s" Sep 11 11:47:49.697965 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:49.697948 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-8fwng" Sep 11 11:47:49.711219 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:49.711203 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-8fwng" Sep 11 11:47:49.774219 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:49.774194 2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 11 11:47:50.190176 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.190148 2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 11 11:47:50.298916 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:50.298879 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf7554308d9b300ebecd820a92d9c922.slice/crio-10ad4721f364e8d5b4e8c32ddf399f59f68a81d9d130a9f4c086ca4ff5912472 WatchSource:0}: Error finding container 10ad4721f364e8d5b4e8c32ddf399f59f68a81d9d130a9f4c086ca4ff5912472: Status 404 returned error can't find the container with id 10ad4721f364e8d5b4e8c32ddf399f59f68a81d9d130a9f4c086ca4ff5912472 Sep 11 11:47:50.299300 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:50.299279 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod579298bbec0b9e8e74fae2628d563376.slice/crio-b88bab130bc62c09f6adc78f2c9138fee23f2e5a1e4968d19bf3da46557563fc WatchSource:0}: Error finding container b88bab130bc62c09f6adc78f2c9138fee23f2e5a1e4968d19bf3da46557563fc: Status 404 returned error can't find the container with id b88bab130bc62c09f6adc78f2c9138fee23f2e5a1e4968d19bf3da46557563fc Sep 11 11:47:50.305045 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.305026 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 11 11:47:50.624819 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.624736 2566 apiserver.go:52] "Watching apiserver" Sep 11 11:47:50.632534 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.632512 2566 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 11 11:47:50.632905 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.632884 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-tfx78","kube-system/kube-apiserver-proxy-ip-10-0-132-170.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hc6pd","openshift-cluster-node-tuning-operator/tuned-fm2g2","openshift-dns/node-resolver-rz4hn","openshift-image-registry/node-ca-2sjcg","openshift-multus/multus-additional-cni-plugins-cs8ww","openshift-network-diagnostics/network-check-target-nq29s","kube-system/konnectivity-agent-n9pc8","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-170.ec2.internal","openshift-multus/multus-bf99t","openshift-multus/network-metrics-daemon-x982g","openshift-network-operator/iptables-alerter-x6tjp","openshift-ovn-kubernetes/ovnkube-node-25t6q"] Sep 11 11:47:50.636702 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.636680 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2sjcg" Sep 11 11:47:50.638752 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.638734 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-x6tjp" Sep 11 11:47:50.639328 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.639304 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ljqjh\"" Sep 11 11:47:50.639422 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.639304 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 11 11:47:50.639422 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.639353 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 11 11:47:50.639422 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.639382 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 11 11:47:50.640854 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.640835 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hc6pd" Sep 11 11:47:50.641340 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.641322 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 11 11:47:50.641475 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.641456 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 11 11:47:50.641558 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.641505 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 11 11:47:50.641638 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.641626 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xdxn9\"" Sep 11 11:47:50.643038 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.643015 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 11 11:47:50.643038 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.643033 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 11 11:47:50.643240 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.643054 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 11 11:47:50.643240 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.643119 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-bgmgc\"" Sep 11 11:47:50.643481 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.643258 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x982g" Sep 11 11:47:50.643481 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:50.643335 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x982g" podUID="d4fd9da2-13c7-4b85-9c89-f10a28d43049" Sep 11 11:47:50.645358 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.645343 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nq29s" Sep 11 11:47:50.645447 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:50.645400 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nq29s" podUID="1598d282-cf11-48e5-b3e5-d3feac72576b" Sep 11 11:47:50.647615 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.647600 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cs8ww" Sep 11 11:47:50.649929 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.649910 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfx78" Sep 11 11:47:50.650007 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:50.649970 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfx78" podUID="2dc78d6d-cf10-4087-a38b-0ded1057f222" Sep 11 11:47:50.650047 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.650020 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 11 11:47:50.650302 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.650285 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 11 11:47:50.650340 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.650286 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 11 11:47:50.650832 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.650816 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 11 11:47:50.650924 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.650855 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 11 11:47:50.650924 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.650905 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xrmth\"" Sep 11 11:47:50.652239 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.652223 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rz4hn" Sep 11 11:47:50.654315 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.654297 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qk8tc\"" Sep 11 11:47:50.654405 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.654389 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-n9pc8" Sep 11 11:47:50.654664 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.654645 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 11 11:47:50.654725 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.654686 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 11 11:47:50.657887 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.656942 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.657887 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.657592 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 11 11:47:50.657887 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.657638 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-xls4w\"" Sep 11 11:47:50.658079 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.657920 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 11 11:47:50.660722 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.660702 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-hx66l\"" Sep 11 11:47:50.660817 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.660744 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 11 11:47:50.660964 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.660947 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.662134 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.662119 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2d1abaa8-2eb8-4feb-9452-a55924a95452-device-dir\") pod \"aws-ebs-csi-driver-node-hc6pd\" (UID: \"2d1abaa8-2eb8-4feb-9452-a55924a95452\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hc6pd" Sep 11 11:47:50.662209 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.662143 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rp9rx\" (UniqueName: \"kubernetes.io/projected/990fcaba-28cc-458a-a2c6-8aaff01fd3ab-kube-api-access-rp9rx\") pod \"iptables-alerter-x6tjp\" (UID: \"990fcaba-28cc-458a-a2c6-8aaff01fd3ab\") " pod="openshift-network-operator/iptables-alerter-x6tjp" Sep 11 11:47:50.662209 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.662161 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2dc78d6d-cf10-4087-a38b-0ded1057f222-original-pull-secret\") pod \"global-pull-secret-syncer-tfx78\" (UID: \"2dc78d6d-cf10-4087-a38b-0ded1057f222\") " pod="kube-system/global-pull-secret-syncer-tfx78" Sep 11 11:47:50.662209 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.662178 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/990fcaba-28cc-458a-a2c6-8aaff01fd3ab-iptables-alerter-script\") pod \"iptables-alerter-x6tjp\" (UID: \"990fcaba-28cc-458a-a2c6-8aaff01fd3ab\") " pod="openshift-network-operator/iptables-alerter-x6tjp" Sep 11 11:47:50.662209 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.662199 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/091f9985-8446-4191-9ec0-9f40b1d1a5fb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cs8ww\" (UID: \"091f9985-8446-4191-9ec0-9f40b1d1a5fb\") " pod="openshift-multus/multus-additional-cni-plugins-cs8ww" Sep 11 11:47:50.662373 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.662281 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/091f9985-8446-4191-9ec0-9f40b1d1a5fb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cs8ww\" (UID: \"091f9985-8446-4191-9ec0-9f40b1d1a5fb\") " pod="openshift-multus/multus-additional-cni-plugins-cs8ww" Sep 11 11:47:50.662373 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.662314 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4fd9da2-13c7-4b85-9c89-f10a28d43049-metrics-certs\") pod \"network-metrics-daemon-x982g\" (UID: \"d4fd9da2-13c7-4b85-9c89-f10a28d43049\") " pod="openshift-multus/network-metrics-daemon-x982g" Sep 11 11:47:50.662373 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.662334 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/091f9985-8446-4191-9ec0-9f40b1d1a5fb-cnibin\") pod \"multus-additional-cni-plugins-cs8ww\" (UID: \"091f9985-8446-4191-9ec0-9f40b1d1a5fb\") " pod="openshift-multus/multus-additional-cni-plugins-cs8ww" Sep 11 11:47:50.662373 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.662348 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5bf5658e-69eb-4ac8-9359-4653f8fd588f-hosts-file\") pod \"node-resolver-rz4hn\" (UID: \"5bf5658e-69eb-4ac8-9359-4653f8fd588f\") " pod="openshift-dns/node-resolver-rz4hn" Sep 11 11:47:50.662499 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.662381 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92f86\" (UniqueName: \"kubernetes.io/projected/5bf5658e-69eb-4ac8-9359-4653f8fd588f-kube-api-access-92f86\") pod \"node-resolver-rz4hn\" (UID: \"5bf5658e-69eb-4ac8-9359-4653f8fd588f\") " pod="openshift-dns/node-resolver-rz4hn" Sep 11 11:47:50.662499 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.662409 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2d1abaa8-2eb8-4feb-9452-a55924a95452-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hc6pd\" (UID: \"2d1abaa8-2eb8-4feb-9452-a55924a95452\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hc6pd" Sep 11 11:47:50.662499 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.662432 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2d1abaa8-2eb8-4feb-9452-a55924a95452-registration-dir\") pod \"aws-ebs-csi-driver-node-hc6pd\" (UID: \"2d1abaa8-2eb8-4feb-9452-a55924a95452\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hc6pd" Sep 11 11:47:50.662499 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.662456 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tfq8\" (UniqueName: \"kubernetes.io/projected/2d1abaa8-2eb8-4feb-9452-a55924a95452-kube-api-access-9tfq8\") pod \"aws-ebs-csi-driver-node-hc6pd\" (UID: \"2d1abaa8-2eb8-4feb-9452-a55924a95452\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hc6pd" Sep 11 11:47:50.662499 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.662481 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2dc78d6d-cf10-4087-a38b-0ded1057f222-dbus\") pod \"global-pull-secret-syncer-tfx78\" (UID: \"2dc78d6d-cf10-4087-a38b-0ded1057f222\") " pod="kube-system/global-pull-secret-syncer-tfx78" Sep 11 11:47:50.662720 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.662504 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f90862ea-2909-4701-ab56-fe2d3fbb83f8-konnectivity-ca\") pod \"konnectivity-agent-n9pc8\" (UID: \"f90862ea-2909-4701-ab56-fe2d3fbb83f8\") " pod="kube-system/konnectivity-agent-n9pc8" Sep 11 11:47:50.662720 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.662579 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2d1abaa8-2eb8-4feb-9452-a55924a95452-etc-selinux\") pod \"aws-ebs-csi-driver-node-hc6pd\" (UID: \"2d1abaa8-2eb8-4feb-9452-a55924a95452\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hc6pd" Sep 11 11:47:50.662720 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.662614 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/091f9985-8446-4191-9ec0-9f40b1d1a5fb-system-cni-dir\") pod \"multus-additional-cni-plugins-cs8ww\" (UID: \"091f9985-8446-4191-9ec0-9f40b1d1a5fb\") " pod="openshift-multus/multus-additional-cni-plugins-cs8ww" Sep 11 11:47:50.662720 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.662641 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e0202693-9deb-4f3b-8d1f-3eec2f3f03d6-host\") pod \"node-ca-2sjcg\" (UID: \"e0202693-9deb-4f3b-8d1f-3eec2f3f03d6\") " pod="openshift-image-registry/node-ca-2sjcg" Sep 11 11:47:50.662720 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.662665 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e0202693-9deb-4f3b-8d1f-3eec2f3f03d6-serviceca\") pod \"node-ca-2sjcg\" (UID: \"e0202693-9deb-4f3b-8d1f-3eec2f3f03d6\") " pod="openshift-image-registry/node-ca-2sjcg" Sep 11 11:47:50.662720 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.662687 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctpz4\" (UniqueName: \"kubernetes.io/projected/e0202693-9deb-4f3b-8d1f-3eec2f3f03d6-kube-api-access-ctpz4\") pod \"node-ca-2sjcg\" (UID: \"e0202693-9deb-4f3b-8d1f-3eec2f3f03d6\") " pod="openshift-image-registry/node-ca-2sjcg" Sep 11 11:47:50.662720 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.662710 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2d1abaa8-2eb8-4feb-9452-a55924a95452-socket-dir\") pod \"aws-ebs-csi-driver-node-hc6pd\" (UID: \"2d1abaa8-2eb8-4feb-9452-a55924a95452\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hc6pd" Sep 11 11:47:50.663053 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.662733 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhdzx\" (UniqueName: \"kubernetes.io/projected/091f9985-8446-4191-9ec0-9f40b1d1a5fb-kube-api-access-vhdzx\") pod \"multus-additional-cni-plugins-cs8ww\" (UID: \"091f9985-8446-4191-9ec0-9f40b1d1a5fb\") " pod="openshift-multus/multus-additional-cni-plugins-cs8ww" Sep 11 11:47:50.663053 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.662758 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5bf5658e-69eb-4ac8-9359-4653f8fd588f-tmp-dir\") pod \"node-resolver-rz4hn\" (UID: \"5bf5658e-69eb-4ac8-9359-4653f8fd588f\") " pod="openshift-dns/node-resolver-rz4hn" Sep 11 11:47:50.663053 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.662783 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2d1abaa8-2eb8-4feb-9452-a55924a95452-sys-fs\") pod \"aws-ebs-csi-driver-node-hc6pd\" (UID: \"2d1abaa8-2eb8-4feb-9452-a55924a95452\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hc6pd" Sep 11 11:47:50.663053 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.662807 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gttx\" (UniqueName: \"kubernetes.io/projected/1598d282-cf11-48e5-b3e5-d3feac72576b-kube-api-access-9gttx\") pod \"network-check-target-nq29s\" (UID: \"1598d282-cf11-48e5-b3e5-d3feac72576b\") " pod="openshift-network-diagnostics/network-check-target-nq29s" Sep 11 11:47:50.663053 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.662829 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/990fcaba-28cc-458a-a2c6-8aaff01fd3ab-host-slash\") pod \"iptables-alerter-x6tjp\" (UID: \"990fcaba-28cc-458a-a2c6-8aaff01fd3ab\") " pod="openshift-network-operator/iptables-alerter-x6tjp" Sep 11 11:47:50.663053 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.662852 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/091f9985-8446-4191-9ec0-9f40b1d1a5fb-cni-binary-copy\") pod \"multus-additional-cni-plugins-cs8ww\" (UID: \"091f9985-8446-4191-9ec0-9f40b1d1a5fb\") " pod="openshift-multus/multus-additional-cni-plugins-cs8ww" Sep 11 11:47:50.663053 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.662941 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/091f9985-8446-4191-9ec0-9f40b1d1a5fb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cs8ww\" (UID: \"091f9985-8446-4191-9ec0-9f40b1d1a5fb\") " pod="openshift-multus/multus-additional-cni-plugins-cs8ww" Sep 11 11:47:50.663053 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.662966 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f90862ea-2909-4701-ab56-fe2d3fbb83f8-agent-certs\") pod \"konnectivity-agent-n9pc8\" (UID: \"f90862ea-2909-4701-ab56-fe2d3fbb83f8\") " pod="kube-system/konnectivity-agent-n9pc8" Sep 11 11:47:50.663053 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.662990 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4xjj\" (UniqueName: \"kubernetes.io/projected/d4fd9da2-13c7-4b85-9c89-f10a28d43049-kube-api-access-p4xjj\") pod \"network-metrics-daemon-x982g\" (UID: \"d4fd9da2-13c7-4b85-9c89-f10a28d43049\") " pod="openshift-multus/network-metrics-daemon-x982g" Sep 11 11:47:50.663053 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.663012 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2dc78d6d-cf10-4087-a38b-0ded1057f222-kubelet-config\") pod \"global-pull-secret-syncer-tfx78\" (UID: \"2dc78d6d-cf10-4087-a38b-0ded1057f222\") " pod="kube-system/global-pull-secret-syncer-tfx78" Sep 11 11:47:50.663583 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.663447 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.663583 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.663468 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-662nm\"" Sep 11 11:47:50.663583 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.663534 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 11 11:47:50.663726 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.663603 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 11 11:47:50.666455 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.666435 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 11 11:47:50.668628 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.668569 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 11 11:47:50.668628 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.668620 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 11 11:47:50.668751 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.668570 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-g6p6d\"" Sep 11 11:47:50.668751 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.668582 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 11 11:47:50.668751 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.668588 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 11 11:47:50.668928 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.668767 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 11 11:47:50.714419 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.714377 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 11:42:49 +0000 UTC" deadline="2028-05-08 10:18:36.735666439 +0000 UTC" Sep 11 11:47:50.714419 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.714413 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14518h30m46.021256284s" Sep 11 11:47:50.752911 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.752891 2566 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 11 11:47:50.755847 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.755811 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-170.ec2.internal" event={"ID":"579298bbec0b9e8e74fae2628d563376","Type":"ContainerStarted","Data":"b88bab130bc62c09f6adc78f2c9138fee23f2e5a1e4968d19bf3da46557563fc"} Sep 11 11:47:50.756887 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.756852 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-170.ec2.internal" event={"ID":"bf7554308d9b300ebecd820a92d9c922","Type":"ContainerStarted","Data":"10ad4721f364e8d5b4e8c32ddf399f59f68a81d9d130a9f4c086ca4ff5912472"} Sep 11 11:47:50.763567 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.763544 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/990fcaba-28cc-458a-a2c6-8aaff01fd3ab-iptables-alerter-script\") pod \"iptables-alerter-x6tjp\" (UID: \"990fcaba-28cc-458a-a2c6-8aaff01fd3ab\") " pod="openshift-network-operator/iptables-alerter-x6tjp" Sep 11 11:47:50.763619 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.763577 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/091f9985-8446-4191-9ec0-9f40b1d1a5fb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cs8ww\" (UID: \"091f9985-8446-4191-9ec0-9f40b1d1a5fb\") " pod="openshift-multus/multus-additional-cni-plugins-cs8ww" Sep 11 11:47:50.763656 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.763600 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/091f9985-8446-4191-9ec0-9f40b1d1a5fb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cs8ww\" (UID: \"091f9985-8446-4191-9ec0-9f40b1d1a5fb\") " pod="openshift-multus/multus-additional-cni-plugins-cs8ww" Sep 11 11:47:50.763743 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.763723 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-log-socket\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.763832 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.763785 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-etc-modprobe-d\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.763887 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.763854 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4fd9da2-13c7-4b85-9c89-f10a28d43049-metrics-certs\") pod \"network-metrics-daemon-x982g\" (UID: \"d4fd9da2-13c7-4b85-9c89-f10a28d43049\") " pod="openshift-multus/network-metrics-daemon-x982g" Sep 11 11:47:50.763930 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.763894 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/091f9985-8446-4191-9ec0-9f40b1d1a5fb-cnibin\") pod \"multus-additional-cni-plugins-cs8ww\" (UID: \"091f9985-8446-4191-9ec0-9f40b1d1a5fb\") " pod="openshift-multus/multus-additional-cni-plugins-cs8ww" Sep 11 11:47:50.763976 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.763935 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92f86\" (UniqueName: \"kubernetes.io/projected/5bf5658e-69eb-4ac8-9359-4653f8fd588f-kube-api-access-92f86\") pod \"node-resolver-rz4hn\" (UID: \"5bf5658e-69eb-4ac8-9359-4653f8fd588f\") " pod="openshift-dns/node-resolver-rz4hn" Sep 11 11:47:50.763976 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.763969 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c48ecd03-9f9f-4c11-95f3-b7400599b444-ovnkube-config\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.764066 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.764025 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2d1abaa8-2eb8-4feb-9452-a55924a95452-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hc6pd\" (UID: \"2d1abaa8-2eb8-4feb-9452-a55924a95452\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hc6pd" Sep 11 11:47:50.764066 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.764017 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/091f9985-8446-4191-9ec0-9f40b1d1a5fb-cnibin\") pod \"multus-additional-cni-plugins-cs8ww\" (UID: \"091f9985-8446-4191-9ec0-9f40b1d1a5fb\") " pod="openshift-multus/multus-additional-cni-plugins-cs8ww" Sep 11 11:47:50.764149 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.764061 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.764149 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:50.764107 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 11:47:50.764149 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.764124 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-cnibin\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.764244 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.764142 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2d1abaa8-2eb8-4feb-9452-a55924a95452-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hc6pd\" (UID: \"2d1abaa8-2eb8-4feb-9452-a55924a95452\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hc6pd" Sep 11 11:47:50.764307 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.764286 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/990fcaba-28cc-458a-a2c6-8aaff01fd3ab-iptables-alerter-script\") pod \"iptables-alerter-x6tjp\" (UID: \"990fcaba-28cc-458a-a2c6-8aaff01fd3ab\") " pod="openshift-network-operator/iptables-alerter-x6tjp" Sep 11 11:47:50.764676 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.764657 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/091f9985-8446-4191-9ec0-9f40b1d1a5fb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cs8ww\" (UID: \"091f9985-8446-4191-9ec0-9f40b1d1a5fb\") " pod="openshift-multus/multus-additional-cni-plugins-cs8ww" Sep 11 11:47:50.764717 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:50.764667 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4fd9da2-13c7-4b85-9c89-f10a28d43049-metrics-certs podName:d4fd9da2-13c7-4b85-9c89-f10a28d43049 nodeName:}" failed. No retries permitted until 2026-09-11 11:47:51.264159406 +0000 UTC m=+3.029132573 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4fd9da2-13c7-4b85-9c89-f10a28d43049-metrics-certs") pod "network-metrics-daemon-x982g" (UID: "d4fd9da2-13c7-4b85-9c89-f10a28d43049") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 11:47:50.764717 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.764704 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-etc-kubernetes\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.764801 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.764722 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-lib-modules\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.764801 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.764736 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-var-lib-kubelet\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.764801 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.764758 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/091f9985-8446-4191-9ec0-9f40b1d1a5fb-system-cni-dir\") pod \"multus-additional-cni-plugins-cs8ww\" (UID: \"091f9985-8446-4191-9ec0-9f40b1d1a5fb\") " pod="openshift-multus/multus-additional-cni-plugins-cs8ww" Sep 11 11:47:50.764935 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.764785 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-var-lib-openvswitch\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.764935 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.764815 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/091f9985-8446-4191-9ec0-9f40b1d1a5fb-system-cni-dir\") pod \"multus-additional-cni-plugins-cs8ww\" (UID: \"091f9985-8446-4191-9ec0-9f40b1d1a5fb\") " pod="openshift-multus/multus-additional-cni-plugins-cs8ww" Sep 11 11:47:50.764935 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.764878 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-host-cni-bin\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.764935 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.764912 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c48ecd03-9f9f-4c11-95f3-b7400599b444-env-overrides\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.765082 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.764936 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-multus-socket-dir-parent\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.765082 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.764960 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgnwg\" (UniqueName: \"kubernetes.io/projected/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-kube-api-access-dgnwg\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.765082 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.764966 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/091f9985-8446-4191-9ec0-9f40b1d1a5fb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cs8ww\" (UID: \"091f9985-8446-4191-9ec0-9f40b1d1a5fb\") " pod="openshift-multus/multus-additional-cni-plugins-cs8ww" Sep 11 11:47:50.765082 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.764991 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e0202693-9deb-4f3b-8d1f-3eec2f3f03d6-host\") pod \"node-ca-2sjcg\" (UID: \"e0202693-9deb-4f3b-8d1f-3eec2f3f03d6\") " pod="openshift-image-registry/node-ca-2sjcg" Sep 11 11:47:50.765082 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.765014 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctpz4\" (UniqueName: \"kubernetes.io/projected/e0202693-9deb-4f3b-8d1f-3eec2f3f03d6-kube-api-access-ctpz4\") pod \"node-ca-2sjcg\" (UID: \"e0202693-9deb-4f3b-8d1f-3eec2f3f03d6\") " pod="openshift-image-registry/node-ca-2sjcg" Sep 11 11:47:50.765082 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.765031 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2d1abaa8-2eb8-4feb-9452-a55924a95452-socket-dir\") pod \"aws-ebs-csi-driver-node-hc6pd\" (UID: \"2d1abaa8-2eb8-4feb-9452-a55924a95452\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hc6pd" Sep 11 11:47:50.765082 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.765064 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhdzx\" (UniqueName: \"kubernetes.io/projected/091f9985-8446-4191-9ec0-9f40b1d1a5fb-kube-api-access-vhdzx\") pod \"multus-additional-cni-plugins-cs8ww\" (UID: \"091f9985-8446-4191-9ec0-9f40b1d1a5fb\") " pod="openshift-multus/multus-additional-cni-plugins-cs8ww" Sep 11 11:47:50.765309 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.765079 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e0202693-9deb-4f3b-8d1f-3eec2f3f03d6-host\") pod \"node-ca-2sjcg\" (UID: \"e0202693-9deb-4f3b-8d1f-3eec2f3f03d6\") " pod="openshift-image-registry/node-ca-2sjcg" Sep 11 11:47:50.765309 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.765109 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-etc-openvswitch\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.765309 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.765157 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-cni-binary-copy\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.765309 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.765158 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2d1abaa8-2eb8-4feb-9452-a55924a95452-socket-dir\") pod \"aws-ebs-csi-driver-node-hc6pd\" (UID: \"2d1abaa8-2eb8-4feb-9452-a55924a95452\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hc6pd" Sep 11 11:47:50.765309 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.765209 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-etc-kubernetes\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.765309 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.765250 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-etc-sysconfig\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.765309 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.765266 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2d1abaa8-2eb8-4feb-9452-a55924a95452-sys-fs\") pod \"aws-ebs-csi-driver-node-hc6pd\" (UID: \"2d1abaa8-2eb8-4feb-9452-a55924a95452\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hc6pd" Sep 11 11:47:50.765309 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.765291 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/990fcaba-28cc-458a-a2c6-8aaff01fd3ab-host-slash\") pod \"iptables-alerter-x6tjp\" (UID: \"990fcaba-28cc-458a-a2c6-8aaff01fd3ab\") " pod="openshift-network-operator/iptables-alerter-x6tjp" Sep 11 11:47:50.765657 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.765340 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f90862ea-2909-4701-ab56-fe2d3fbb83f8-agent-certs\") pod \"konnectivity-agent-n9pc8\" (UID: \"f90862ea-2909-4701-ab56-fe2d3fbb83f8\") " pod="kube-system/konnectivity-agent-n9pc8" Sep 11 11:47:50.765657 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.765382 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-system-cni-dir\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.765657 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.765394 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/990fcaba-28cc-458a-a2c6-8aaff01fd3ab-host-slash\") pod \"iptables-alerter-x6tjp\" (UID: \"990fcaba-28cc-458a-a2c6-8aaff01fd3ab\") " pod="openshift-network-operator/iptables-alerter-x6tjp" Sep 11 11:47:50.765657 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.765414 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-etc-tuned\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.765657 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.765441 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2d1abaa8-2eb8-4feb-9452-a55924a95452-sys-fs\") pod \"aws-ebs-csi-driver-node-hc6pd\" (UID: \"2d1abaa8-2eb8-4feb-9452-a55924a95452\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hc6pd" Sep 11 11:47:50.765657 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.765449 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-tmp\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.765657 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.765481 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-host-kubelet\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.765657 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.765501 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-run-ovn\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.765657 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.765536 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-multus-cni-dir\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.765657 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.765579 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-etc-sysctl-d\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.765657 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.765624 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rp9rx\" (UniqueName: \"kubernetes.io/projected/990fcaba-28cc-458a-a2c6-8aaff01fd3ab-kube-api-access-rp9rx\") pod \"iptables-alerter-x6tjp\" (UID: \"990fcaba-28cc-458a-a2c6-8aaff01fd3ab\") " pod="openshift-network-operator/iptables-alerter-x6tjp" Sep 11 11:47:50.766132 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.765669 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2dc78d6d-cf10-4087-a38b-0ded1057f222-original-pull-secret\") pod \"global-pull-secret-syncer-tfx78\" (UID: \"2dc78d6d-cf10-4087-a38b-0ded1057f222\") " pod="kube-system/global-pull-secret-syncer-tfx78" Sep 11 11:47:50.766132 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.765690 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-host-run-netns\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.766132 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.765727 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c48ecd03-9f9f-4c11-95f3-b7400599b444-ovn-node-metrics-cert\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.766132 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.765780 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-host-var-lib-cni-bin\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.766132 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:50.765795 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 11 11:47:50.766132 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.765820 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-host-slash\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.766132 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:50.765896 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2dc78d6d-cf10-4087-a38b-0ded1057f222-original-pull-secret podName:2dc78d6d-cf10-4087-a38b-0ded1057f222 nodeName:}" failed. No retries permitted until 2026-09-11 11:47:51.265850358 +0000 UTC m=+3.030823515 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2dc78d6d-cf10-4087-a38b-0ded1057f222-original-pull-secret") pod "global-pull-secret-syncer-tfx78" (UID: "2dc78d6d-cf10-4087-a38b-0ded1057f222") : object "kube-system"/"original-pull-secret" not registered Sep 11 11:47:50.766132 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.765925 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zkll\" (UniqueName: \"kubernetes.io/projected/c48ecd03-9f9f-4c11-95f3-b7400599b444-kube-api-access-9zkll\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.766132 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.765926 2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 11 11:47:50.766132 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.765944 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-multus-conf-dir\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.766132 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.765988 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-run\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.766132 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.766013 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5bf5658e-69eb-4ac8-9359-4653f8fd588f-hosts-file\") pod \"node-resolver-rz4hn\" (UID: \"5bf5658e-69eb-4ac8-9359-4653f8fd588f\") " pod="openshift-dns/node-resolver-rz4hn" Sep 11 11:47:50.766132 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.766035 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-run-openvswitch\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.766132 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.766054 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-node-log\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.766132 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.766067 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-hostroot\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.766132 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.766084 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-multus-daemon-config\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.766132 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.766141 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5bf5658e-69eb-4ac8-9359-4653f8fd588f-hosts-file\") pod \"node-resolver-rz4hn\" (UID: \"5bf5658e-69eb-4ac8-9359-4653f8fd588f\") " pod="openshift-dns/node-resolver-rz4hn" Sep 11 11:47:50.766723 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.766162 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2d1abaa8-2eb8-4feb-9452-a55924a95452-registration-dir\") pod \"aws-ebs-csi-driver-node-hc6pd\" (UID: \"2d1abaa8-2eb8-4feb-9452-a55924a95452\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hc6pd" Sep 11 11:47:50.766723 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.766187 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tfq8\" (UniqueName: \"kubernetes.io/projected/2d1abaa8-2eb8-4feb-9452-a55924a95452-kube-api-access-9tfq8\") pod \"aws-ebs-csi-driver-node-hc6pd\" (UID: \"2d1abaa8-2eb8-4feb-9452-a55924a95452\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hc6pd" Sep 11 11:47:50.766723 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.766229 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2d1abaa8-2eb8-4feb-9452-a55924a95452-registration-dir\") pod \"aws-ebs-csi-driver-node-hc6pd\" (UID: \"2d1abaa8-2eb8-4feb-9452-a55924a95452\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hc6pd" Sep 11 11:47:50.766723 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.766244 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2dc78d6d-cf10-4087-a38b-0ded1057f222-dbus\") pod \"global-pull-secret-syncer-tfx78\" (UID: \"2dc78d6d-cf10-4087-a38b-0ded1057f222\") " pod="kube-system/global-pull-secret-syncer-tfx78" Sep 11 11:47:50.766723 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.766268 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f90862ea-2909-4701-ab56-fe2d3fbb83f8-konnectivity-ca\") pod \"konnectivity-agent-n9pc8\" (UID: \"f90862ea-2909-4701-ab56-fe2d3fbb83f8\") " pod="kube-system/konnectivity-agent-n9pc8" Sep 11 11:47:50.766723 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.766322 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c48ecd03-9f9f-4c11-95f3-b7400599b444-ovnkube-script-lib\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.766723 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.766348 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-host-run-netns\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.766723 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.766339 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2dc78d6d-cf10-4087-a38b-0ded1057f222-dbus\") pod \"global-pull-secret-syncer-tfx78\" (UID: \"2dc78d6d-cf10-4087-a38b-0ded1057f222\") " pod="kube-system/global-pull-secret-syncer-tfx78" Sep 11 11:47:50.766723 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.766379 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-host-var-lib-cni-multus\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.766723 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.766411 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2d1abaa8-2eb8-4feb-9452-a55924a95452-etc-selinux\") pod \"aws-ebs-csi-driver-node-hc6pd\" (UID: \"2d1abaa8-2eb8-4feb-9452-a55924a95452\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hc6pd" Sep 11 11:47:50.766723 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.766474 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-host-cni-netd\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.766723 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.766516 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2d1abaa8-2eb8-4feb-9452-a55924a95452-etc-selinux\") pod \"aws-ebs-csi-driver-node-hc6pd\" (UID: \"2d1abaa8-2eb8-4feb-9452-a55924a95452\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hc6pd" Sep 11 11:47:50.766723 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.766578 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-host-run-k8s-cni-cncf-io\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.766723 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.766611 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-host\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.766723 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.766665 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e0202693-9deb-4f3b-8d1f-3eec2f3f03d6-serviceca\") pod \"node-ca-2sjcg\" (UID: \"e0202693-9deb-4f3b-8d1f-3eec2f3f03d6\") " pod="openshift-image-registry/node-ca-2sjcg" Sep 11 11:47:50.766723 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.766689 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5bf5658e-69eb-4ac8-9359-4653f8fd588f-tmp-dir\") pod \"node-resolver-rz4hn\" (UID: \"5bf5658e-69eb-4ac8-9359-4653f8fd588f\") " pod="openshift-dns/node-resolver-rz4hn" Sep 11 11:47:50.766723 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.766702 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-host-var-lib-kubelet\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.767673 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.766720 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktnnh\" (UniqueName: \"kubernetes.io/projected/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-kube-api-access-ktnnh\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.767673 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.766746 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gttx\" (UniqueName: \"kubernetes.io/projected/1598d282-cf11-48e5-b3e5-d3feac72576b-kube-api-access-9gttx\") pod \"network-check-target-nq29s\" (UID: \"1598d282-cf11-48e5-b3e5-d3feac72576b\") " pod="openshift-network-diagnostics/network-check-target-nq29s" Sep 11 11:47:50.767673 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.766795 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/091f9985-8446-4191-9ec0-9f40b1d1a5fb-cni-binary-copy\") pod \"multus-additional-cni-plugins-cs8ww\" (UID: \"091f9985-8446-4191-9ec0-9f40b1d1a5fb\") " pod="openshift-multus/multus-additional-cni-plugins-cs8ww" Sep 11 11:47:50.767673 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.766818 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/091f9985-8446-4191-9ec0-9f40b1d1a5fb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cs8ww\" (UID: \"091f9985-8446-4191-9ec0-9f40b1d1a5fb\") " pod="openshift-multus/multus-additional-cni-plugins-cs8ww" Sep 11 11:47:50.767673 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.766896 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-host-run-ovn-kubernetes\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.767673 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.766927 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4xjj\" (UniqueName: \"kubernetes.io/projected/d4fd9da2-13c7-4b85-9c89-f10a28d43049-kube-api-access-p4xjj\") pod \"network-metrics-daemon-x982g\" (UID: \"d4fd9da2-13c7-4b85-9c89-f10a28d43049\") " pod="openshift-multus/network-metrics-daemon-x982g" Sep 11 11:47:50.767673 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.766966 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2dc78d6d-cf10-4087-a38b-0ded1057f222-kubelet-config\") pod \"global-pull-secret-syncer-tfx78\" (UID: \"2dc78d6d-cf10-4087-a38b-0ded1057f222\") " pod="kube-system/global-pull-secret-syncer-tfx78" Sep 11 11:47:50.767673 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.767059 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f90862ea-2909-4701-ab56-fe2d3fbb83f8-konnectivity-ca\") pod \"konnectivity-agent-n9pc8\" (UID: \"f90862ea-2909-4701-ab56-fe2d3fbb83f8\") " pod="kube-system/konnectivity-agent-n9pc8" Sep 11 11:47:50.767673 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.767067 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5bf5658e-69eb-4ac8-9359-4653f8fd588f-tmp-dir\") pod \"node-resolver-rz4hn\" (UID: \"5bf5658e-69eb-4ac8-9359-4653f8fd588f\") " pod="openshift-dns/node-resolver-rz4hn" Sep 11 11:47:50.767673 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.767060 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-run-systemd\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.767673 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.767150 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-host-run-multus-certs\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.767673 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.767172 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-sys\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.767673 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.767199 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2d1abaa8-2eb8-4feb-9452-a55924a95452-device-dir\") pod \"aws-ebs-csi-driver-node-hc6pd\" (UID: \"2d1abaa8-2eb8-4feb-9452-a55924a95452\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hc6pd" Sep 11 11:47:50.767673 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.767253 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-systemd-units\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.767673 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.767281 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-etc-sysctl-conf\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.767673 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.767304 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-etc-systemd\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.767673 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.767334 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/091f9985-8446-4191-9ec0-9f40b1d1a5fb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cs8ww\" (UID: \"091f9985-8446-4191-9ec0-9f40b1d1a5fb\") " pod="openshift-multus/multus-additional-cni-plugins-cs8ww" Sep 11 11:47:50.768253 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.767170 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e0202693-9deb-4f3b-8d1f-3eec2f3f03d6-serviceca\") pod \"node-ca-2sjcg\" (UID: \"e0202693-9deb-4f3b-8d1f-3eec2f3f03d6\") " pod="openshift-image-registry/node-ca-2sjcg" Sep 11 11:47:50.768253 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.767455 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2dc78d6d-cf10-4087-a38b-0ded1057f222-kubelet-config\") pod \"global-pull-secret-syncer-tfx78\" (UID: \"2dc78d6d-cf10-4087-a38b-0ded1057f222\") " pod="kube-system/global-pull-secret-syncer-tfx78" Sep 11 11:47:50.768253 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.767474 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2d1abaa8-2eb8-4feb-9452-a55924a95452-device-dir\") pod \"aws-ebs-csi-driver-node-hc6pd\" (UID: \"2d1abaa8-2eb8-4feb-9452-a55924a95452\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hc6pd" Sep 11 11:47:50.768253 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.767551 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/091f9985-8446-4191-9ec0-9f40b1d1a5fb-cni-binary-copy\") pod \"multus-additional-cni-plugins-cs8ww\" (UID: \"091f9985-8446-4191-9ec0-9f40b1d1a5fb\") " pod="openshift-multus/multus-additional-cni-plugins-cs8ww" Sep 11 11:47:50.769991 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.769973 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f90862ea-2909-4701-ab56-fe2d3fbb83f8-agent-certs\") pod \"konnectivity-agent-n9pc8\" (UID: \"f90862ea-2909-4701-ab56-fe2d3fbb83f8\") " pod="kube-system/konnectivity-agent-n9pc8" Sep 11 11:47:50.774522 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:50.774502 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 11 11:47:50.774622 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:50.774527 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 11 11:47:50.774622 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:50.774541 2566 projected.go:196] Error preparing data for projected volume kube-api-access-9gttx for pod openshift-network-diagnostics/network-check-target-nq29s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 11:47:50.774622 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:50.774598 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1598d282-cf11-48e5-b3e5-d3feac72576b-kube-api-access-9gttx podName:1598d282-cf11-48e5-b3e5-d3feac72576b nodeName:}" failed. No retries permitted until 2026-09-11 11:47:51.274582878 +0000 UTC m=+3.039556034 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-9gttx" (UniqueName: "kubernetes.io/projected/1598d282-cf11-48e5-b3e5-d3feac72576b-kube-api-access-9gttx") pod "network-check-target-nq29s" (UID: "1598d282-cf11-48e5-b3e5-d3feac72576b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 11:47:50.776153 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.776127 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92f86\" (UniqueName: \"kubernetes.io/projected/5bf5658e-69eb-4ac8-9359-4653f8fd588f-kube-api-access-92f86\") pod \"node-resolver-rz4hn\" (UID: \"5bf5658e-69eb-4ac8-9359-4653f8fd588f\") " pod="openshift-dns/node-resolver-rz4hn" Sep 11 11:47:50.776883 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.776846 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhdzx\" (UniqueName: \"kubernetes.io/projected/091f9985-8446-4191-9ec0-9f40b1d1a5fb-kube-api-access-vhdzx\") pod \"multus-additional-cni-plugins-cs8ww\" (UID: \"091f9985-8446-4191-9ec0-9f40b1d1a5fb\") " pod="openshift-multus/multus-additional-cni-plugins-cs8ww" Sep 11 11:47:50.778413 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.778363 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctpz4\" (UniqueName: \"kubernetes.io/projected/e0202693-9deb-4f3b-8d1f-3eec2f3f03d6-kube-api-access-ctpz4\") pod \"node-ca-2sjcg\" (UID: \"e0202693-9deb-4f3b-8d1f-3eec2f3f03d6\") " pod="openshift-image-registry/node-ca-2sjcg" Sep 11 11:47:50.779038 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.779020 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tfq8\" (UniqueName: \"kubernetes.io/projected/2d1abaa8-2eb8-4feb-9452-a55924a95452-kube-api-access-9tfq8\") pod \"aws-ebs-csi-driver-node-hc6pd\" (UID: \"2d1abaa8-2eb8-4feb-9452-a55924a95452\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hc6pd" Sep 11 11:47:50.779799 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.779773 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rp9rx\" (UniqueName: \"kubernetes.io/projected/990fcaba-28cc-458a-a2c6-8aaff01fd3ab-kube-api-access-rp9rx\") pod \"iptables-alerter-x6tjp\" (UID: \"990fcaba-28cc-458a-a2c6-8aaff01fd3ab\") " pod="openshift-network-operator/iptables-alerter-x6tjp" Sep 11 11:47:50.780981 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.780965 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4xjj\" (UniqueName: \"kubernetes.io/projected/d4fd9da2-13c7-4b85-9c89-f10a28d43049-kube-api-access-p4xjj\") pod \"network-metrics-daemon-x982g\" (UID: \"d4fd9da2-13c7-4b85-9c89-f10a28d43049\") " pod="openshift-multus/network-metrics-daemon-x982g" Sep 11 11:47:50.835723 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.835693 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 11 11:47:50.867730 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.867697 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-run-openvswitch\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.867956 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.867735 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-node-log\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.867956 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.867760 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-hostroot\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.867956 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.867797 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-multus-daemon-config\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.867956 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.867807 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-node-log\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.867956 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.867809 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-run-openvswitch\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.867956 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.867859 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-hostroot\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.867956 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.867903 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c48ecd03-9f9f-4c11-95f3-b7400599b444-ovnkube-script-lib\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.867956 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.867938 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-host-run-netns\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.867956 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.867959 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-host-var-lib-cni-multus\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.868392 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.867983 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-host-cni-netd\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.868392 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868011 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-host-run-netns\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.868392 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868053 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-host-var-lib-cni-multus\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.868392 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868059 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-host-run-k8s-cni-cncf-io\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.868392 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868016 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-host-run-k8s-cni-cncf-io\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.868392 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868098 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-host\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.868392 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868125 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-host-var-lib-kubelet\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.868392 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868151 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktnnh\" (UniqueName: \"kubernetes.io/projected/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-kube-api-access-ktnnh\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.868392 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868191 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-host-run-ovn-kubernetes\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.868392 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868209 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-host\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.868392 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868219 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-run-systemd\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.868392 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868241 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-host-run-multus-certs\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.868392 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868263 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-sys\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.868392 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868285 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-systemd-units\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.868392 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868269 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-host-var-lib-kubelet\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.868392 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868308 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-etc-sysctl-conf\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.868392 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868341 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-etc-systemd\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.868392 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868380 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-multus-daemon-config\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.869238 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868404 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-log-socket\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.869238 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868436 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-etc-sysctl-conf\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.869238 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868440 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-run-systemd\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.869238 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868097 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-host-cni-netd\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.869238 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868462 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-sys\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.869238 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868461 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c48ecd03-9f9f-4c11-95f3-b7400599b444-ovnkube-script-lib\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.869238 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868479 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-systemd-units\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.869238 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868495 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-host-run-ovn-kubernetes\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.869238 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868511 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-host-run-multus-certs\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.869238 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868520 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-log-socket\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.869238 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868533 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-etc-modprobe-d\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.869238 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868580 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c48ecd03-9f9f-4c11-95f3-b7400599b444-ovnkube-config\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.869238 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868532 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-etc-systemd\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.869238 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868608 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.869238 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868636 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-cnibin\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.869238 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868643 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-etc-modprobe-d\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.869238 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868665 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.870137 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868659 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-etc-kubernetes\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.870137 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868706 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-cnibin\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.870137 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868709 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-lib-modules\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.870137 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868750 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-var-lib-kubelet\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.870137 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868761 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-etc-kubernetes\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.870137 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868773 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-var-lib-openvswitch\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.870137 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868793 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-lib-modules\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.870137 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868795 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-host-cni-bin\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.870137 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868809 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-var-lib-kubelet\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.870137 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868822 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-host-cni-bin\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.870137 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868826 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c48ecd03-9f9f-4c11-95f3-b7400599b444-env-overrides\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.870137 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868849 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-var-lib-openvswitch\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.870137 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868852 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-multus-socket-dir-parent\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.870137 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868897 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgnwg\" (UniqueName: \"kubernetes.io/projected/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-kube-api-access-dgnwg\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.870137 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868927 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-etc-openvswitch\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.870137 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868952 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-cni-binary-copy\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.870137 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868975 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-etc-kubernetes\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.870137 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.868984 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-multus-socket-dir-parent\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.871003 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.869001 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-etc-sysconfig\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.871003 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.869028 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-system-cni-dir\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.871003 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.869050 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-etc-tuned\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.871003 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.869072 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-tmp\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.871003 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.869095 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-host-kubelet\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.871003 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.869118 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-run-ovn\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.871003 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.869123 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-system-cni-dir\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.871003 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.869163 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-run-ovn\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.871003 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.869185 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c48ecd03-9f9f-4c11-95f3-b7400599b444-env-overrides\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.871003 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.869190 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-multus-cni-dir\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.871003 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.869072 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c48ecd03-9f9f-4c11-95f3-b7400599b444-ovnkube-config\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.871003 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.869225 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-etc-openvswitch\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.871003 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.869220 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-etc-sysctl-d\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.871003 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.869246 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-host-kubelet\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.871003 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.869268 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-host-run-netns\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.871003 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.869279 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-multus-cni-dir\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.871003 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.869283 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c48ecd03-9f9f-4c11-95f3-b7400599b444-ovn-node-metrics-cert\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.871003 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.869184 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-etc-sysconfig\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.871826 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.869300 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-host-var-lib-cni-bin\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.871826 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.869331 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-host-var-lib-cni-bin\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.871826 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.869331 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-host-slash\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.871826 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.869344 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-etc-sysctl-d\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.871826 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.869361 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zkll\" (UniqueName: \"kubernetes.io/projected/c48ecd03-9f9f-4c11-95f3-b7400599b444-kube-api-access-9zkll\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.871826 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.869386 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-multus-conf-dir\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.871826 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.869393 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-host-slash\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.871826 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.869409 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-run\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.871826 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.869445 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c48ecd03-9f9f-4c11-95f3-b7400599b444-host-run-netns\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.871826 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.869449 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-run\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.871826 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.869473 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-multus-conf-dir\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.871826 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.869496 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-cni-binary-copy\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.871826 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.869619 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-etc-kubernetes\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.872430 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.871856 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-etc-tuned\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.873158 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.872783 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-tmp\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.873158 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.873115 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c48ecd03-9f9f-4c11-95f3-b7400599b444-ovn-node-metrics-cert\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.877981 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.877923 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgnwg\" (UniqueName: \"kubernetes.io/projected/e7a5ff3b-7309-483c-bc1f-3b0d86c2216e-kube-api-access-dgnwg\") pod \"tuned-fm2g2\" (UID: \"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e\") " pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.879478 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.879457 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zkll\" (UniqueName: \"kubernetes.io/projected/c48ecd03-9f9f-4c11-95f3-b7400599b444-kube-api-access-9zkll\") pod \"ovnkube-node-25t6q\" (UID: \"c48ecd03-9f9f-4c11-95f3-b7400599b444\") " pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.879806 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.879779 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktnnh\" (UniqueName: \"kubernetes.io/projected/1f35e6ed-521d-436d-8bf6-ee60e8d8fa05-kube-api-access-ktnnh\") pod \"multus-bf99t\" (UID: \"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05\") " pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.946731 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.946691 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2sjcg" Sep 11 11:47:50.953706 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.953681 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-x6tjp" Sep 11 11:47:50.961613 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.961574 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hc6pd" Sep 11 11:47:50.965495 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:50.965468 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod990fcaba_28cc_458a_a2c6_8aaff01fd3ab.slice/crio-c72e834d00b26db2166e59b1f3c184404c301e811b8c06902b1c6d4b3a211e18 WatchSource:0}: Error finding container c72e834d00b26db2166e59b1f3c184404c301e811b8c06902b1c6d4b3a211e18: Status 404 returned error can't find the container with id c72e834d00b26db2166e59b1f3c184404c301e811b8c06902b1c6d4b3a211e18 Sep 11 11:47:50.966658 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.966635 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cs8ww" Sep 11 11:47:50.971462 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.971320 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rz4hn" Sep 11 11:47:50.975595 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:50.975567 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d1abaa8_2eb8_4feb_9452_a55924a95452.slice/crio-39486981b1c210f4aba45e070f137b26951cb636e5fdfe5ad68b21b50b44806e WatchSource:0}: Error finding container 39486981b1c210f4aba45e070f137b26951cb636e5fdfe5ad68b21b50b44806e: Status 404 returned error can't find the container with id 39486981b1c210f4aba45e070f137b26951cb636e5fdfe5ad68b21b50b44806e Sep 11 11:47:50.977017 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.976997 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-n9pc8" Sep 11 11:47:50.978229 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:50.978205 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod091f9985_8446_4191_9ec0_9f40b1d1a5fb.slice/crio-cf759a8fdb32786f900d1eb7122c6c6a7ec3276e7d26141c4648ae0af52fc7c8 WatchSource:0}: Error finding container cf759a8fdb32786f900d1eb7122c6c6a7ec3276e7d26141c4648ae0af52fc7c8: Status 404 returned error can't find the container with id cf759a8fdb32786f900d1eb7122c6c6a7ec3276e7d26141c4648ae0af52fc7c8 Sep 11 11:47:50.982520 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.982502 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bf99t" Sep 11 11:47:50.988429 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.988390 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" Sep 11 11:47:50.992770 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:50.992750 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:47:50.994504 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:50.994478 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bf5658e_69eb_4ac8_9359_4653f8fd588f.slice/crio-cd97ca853c62314eca9787a772360077f3368c4db3dea1a06f23e9700bc4a382 WatchSource:0}: Error finding container cd97ca853c62314eca9787a772360077f3368c4db3dea1a06f23e9700bc4a382: Status 404 returned error can't find the container with id cd97ca853c62314eca9787a772360077f3368c4db3dea1a06f23e9700bc4a382 Sep 11 11:47:50.996902 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:50.996881 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf90862ea_2909_4701_ab56_fe2d3fbb83f8.slice/crio-ef568ed0bc8495aa4df23d4f2de2c675ffb46d7864fa4c4377c1d779814d8bcf WatchSource:0}: Error finding container ef568ed0bc8495aa4df23d4f2de2c675ffb46d7864fa4c4377c1d779814d8bcf: Status 404 returned error can't find the container with id ef568ed0bc8495aa4df23d4f2de2c675ffb46d7864fa4c4377c1d779814d8bcf Sep 11 11:47:51.001017 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:51.000818 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7a5ff3b_7309_483c_bc1f_3b0d86c2216e.slice/crio-6efd0753cc39d4935c0c26f792b50f4c84151de345f4c1209051d6ed2ecec920 WatchSource:0}: Error finding container 6efd0753cc39d4935c0c26f792b50f4c84151de345f4c1209051d6ed2ecec920: Status 404 returned error can't find the container with id 6efd0753cc39d4935c0c26f792b50f4c84151de345f4c1209051d6ed2ecec920 Sep 11 11:47:51.008250 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:47:51.008227 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc48ecd03_9f9f_4c11_95f3_b7400599b444.slice/crio-b8896dee28763f1c4884ee65da649358ad399dca528c86daa2249ea8757768e5 WatchSource:0}: Error finding container b8896dee28763f1c4884ee65da649358ad399dca528c86daa2249ea8757768e5: Status 404 returned error can't find the container with id b8896dee28763f1c4884ee65da649358ad399dca528c86daa2249ea8757768e5 Sep 11 11:47:51.273359 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:51.273112 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2dc78d6d-cf10-4087-a38b-0ded1057f222-original-pull-secret\") pod \"global-pull-secret-syncer-tfx78\" (UID: \"2dc78d6d-cf10-4087-a38b-0ded1057f222\") " pod="kube-system/global-pull-secret-syncer-tfx78" Sep 11 11:47:51.273499 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:51.273366 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4fd9da2-13c7-4b85-9c89-f10a28d43049-metrics-certs\") pod \"network-metrics-daemon-x982g\" (UID: \"d4fd9da2-13c7-4b85-9c89-f10a28d43049\") " pod="openshift-multus/network-metrics-daemon-x982g" Sep 11 11:47:51.273499 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:51.273269 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 11 11:47:51.273499 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:51.273468 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2dc78d6d-cf10-4087-a38b-0ded1057f222-original-pull-secret podName:2dc78d6d-cf10-4087-a38b-0ded1057f222 nodeName:}" failed. No retries permitted until 2026-09-11 11:47:52.273444374 +0000 UTC m=+4.038417542 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2dc78d6d-cf10-4087-a38b-0ded1057f222-original-pull-secret") pod "global-pull-secret-syncer-tfx78" (UID: "2dc78d6d-cf10-4087-a38b-0ded1057f222") : object "kube-system"/"original-pull-secret" not registered Sep 11 11:47:51.273499 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:51.273498 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 11:47:51.273712 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:51.273537 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4fd9da2-13c7-4b85-9c89-f10a28d43049-metrics-certs podName:d4fd9da2-13c7-4b85-9c89-f10a28d43049 nodeName:}" failed. No retries permitted until 2026-09-11 11:47:52.273522306 +0000 UTC m=+4.038495460 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4fd9da2-13c7-4b85-9c89-f10a28d43049-metrics-certs") pod "network-metrics-daemon-x982g" (UID: "d4fd9da2-13c7-4b85-9c89-f10a28d43049") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 11:47:51.374506 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:51.374470 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gttx\" (UniqueName: \"kubernetes.io/projected/1598d282-cf11-48e5-b3e5-d3feac72576b-kube-api-access-9gttx\") pod \"network-check-target-nq29s\" (UID: \"1598d282-cf11-48e5-b3e5-d3feac72576b\") " pod="openshift-network-diagnostics/network-check-target-nq29s" Sep 11 11:47:51.374690 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:51.374665 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 11 11:47:51.374748 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:51.374698 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 11 11:47:51.374748 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:51.374712 2566 projected.go:196] Error preparing data for projected volume kube-api-access-9gttx for pod openshift-network-diagnostics/network-check-target-nq29s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 11:47:51.374830 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:51.374781 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1598d282-cf11-48e5-b3e5-d3feac72576b-kube-api-access-9gttx podName:1598d282-cf11-48e5-b3e5-d3feac72576b nodeName:}" failed. No retries permitted until 2026-09-11 11:47:52.374760101 +0000 UTC m=+4.139733264 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-9gttx" (UniqueName: "kubernetes.io/projected/1598d282-cf11-48e5-b3e5-d3feac72576b-kube-api-access-9gttx") pod "network-check-target-nq29s" (UID: "1598d282-cf11-48e5-b3e5-d3feac72576b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 11:47:51.715569 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:51.715469 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 11:42:49 +0000 UTC" deadline="2028-03-22 20:00:56.639834562 +0000 UTC" Sep 11 11:47:51.715569 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:51.715520 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13400h13m4.924317332s" Sep 11 11:47:51.760376 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:51.760337 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cs8ww" event={"ID":"091f9985-8446-4191-9ec0-9f40b1d1a5fb","Type":"ContainerStarted","Data":"cf759a8fdb32786f900d1eb7122c6c6a7ec3276e7d26141c4648ae0af52fc7c8"} Sep 11 11:47:51.761669 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:51.761632 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" event={"ID":"c48ecd03-9f9f-4c11-95f3-b7400599b444","Type":"ContainerStarted","Data":"b8896dee28763f1c4884ee65da649358ad399dca528c86daa2249ea8757768e5"} Sep 11 11:47:51.762855 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:51.762826 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-n9pc8" event={"ID":"f90862ea-2909-4701-ab56-fe2d3fbb83f8","Type":"ContainerStarted","Data":"ef568ed0bc8495aa4df23d4f2de2c675ffb46d7864fa4c4377c1d779814d8bcf"} Sep 11 11:47:51.764163 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:51.764133 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rz4hn" event={"ID":"5bf5658e-69eb-4ac8-9359-4653f8fd588f","Type":"ContainerStarted","Data":"cd97ca853c62314eca9787a772360077f3368c4db3dea1a06f23e9700bc4a382"} Sep 11 11:47:51.765424 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:51.765398 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hc6pd" event={"ID":"2d1abaa8-2eb8-4feb-9452-a55924a95452","Type":"ContainerStarted","Data":"39486981b1c210f4aba45e070f137b26951cb636e5fdfe5ad68b21b50b44806e"} Sep 11 11:47:51.766745 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:51.766724 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-x6tjp" event={"ID":"990fcaba-28cc-458a-a2c6-8aaff01fd3ab","Type":"ContainerStarted","Data":"c72e834d00b26db2166e59b1f3c184404c301e811b8c06902b1c6d4b3a211e18"} Sep 11 11:47:51.767985 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:51.767955 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2sjcg" event={"ID":"e0202693-9deb-4f3b-8d1f-3eec2f3f03d6","Type":"ContainerStarted","Data":"8a549c28b86b34a4170c7dd5ee0c8c7994e6f4cdec54cb0d9b5c68dab62df051"} Sep 11 11:47:51.769221 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:51.769176 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" event={"ID":"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e","Type":"ContainerStarted","Data":"6efd0753cc39d4935c0c26f792b50f4c84151de345f4c1209051d6ed2ecec920"} Sep 11 11:47:51.770383 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:51.770360 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bf99t" event={"ID":"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05","Type":"ContainerStarted","Data":"daa627ed7ccebd84b081c125836299bfe0d39b6c1f03abc1ef6b8452f453404a"} Sep 11 11:47:52.280180 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:52.280136 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4fd9da2-13c7-4b85-9c89-f10a28d43049-metrics-certs\") pod \"network-metrics-daemon-x982g\" (UID: \"d4fd9da2-13c7-4b85-9c89-f10a28d43049\") " pod="openshift-multus/network-metrics-daemon-x982g" Sep 11 11:47:52.280348 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:52.280197 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2dc78d6d-cf10-4087-a38b-0ded1057f222-original-pull-secret\") pod \"global-pull-secret-syncer-tfx78\" (UID: \"2dc78d6d-cf10-4087-a38b-0ded1057f222\") " pod="kube-system/global-pull-secret-syncer-tfx78" Sep 11 11:47:52.280348 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:52.280318 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 11 11:47:52.280474 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:52.280373 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2dc78d6d-cf10-4087-a38b-0ded1057f222-original-pull-secret podName:2dc78d6d-cf10-4087-a38b-0ded1057f222 nodeName:}" failed. No retries permitted until 2026-09-11 11:47:54.280360632 +0000 UTC m=+6.045333800 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2dc78d6d-cf10-4087-a38b-0ded1057f222-original-pull-secret") pod "global-pull-secret-syncer-tfx78" (UID: "2dc78d6d-cf10-4087-a38b-0ded1057f222") : object "kube-system"/"original-pull-secret" not registered Sep 11 11:47:52.280474 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:52.280314 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 11:47:52.280594 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:52.280495 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4fd9da2-13c7-4b85-9c89-f10a28d43049-metrics-certs podName:d4fd9da2-13c7-4b85-9c89-f10a28d43049 nodeName:}" failed. No retries permitted until 2026-09-11 11:47:54.280466114 +0000 UTC m=+6.045439281 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4fd9da2-13c7-4b85-9c89-f10a28d43049-metrics-certs") pod "network-metrics-daemon-x982g" (UID: "d4fd9da2-13c7-4b85-9c89-f10a28d43049") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 11:47:52.381225 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:52.381185 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gttx\" (UniqueName: \"kubernetes.io/projected/1598d282-cf11-48e5-b3e5-d3feac72576b-kube-api-access-9gttx\") pod \"network-check-target-nq29s\" (UID: \"1598d282-cf11-48e5-b3e5-d3feac72576b\") " pod="openshift-network-diagnostics/network-check-target-nq29s" Sep 11 11:47:52.381406 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:52.381354 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 11 11:47:52.381406 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:52.381376 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 11 11:47:52.381406 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:52.381389 2566 projected.go:196] Error preparing data for projected volume kube-api-access-9gttx for pod openshift-network-diagnostics/network-check-target-nq29s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 11:47:52.381565 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:52.381448 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1598d282-cf11-48e5-b3e5-d3feac72576b-kube-api-access-9gttx podName:1598d282-cf11-48e5-b3e5-d3feac72576b nodeName:}" failed. No retries permitted until 2026-09-11 11:47:54.381429797 +0000 UTC m=+6.146402952 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-9gttx" (UniqueName: "kubernetes.io/projected/1598d282-cf11-48e5-b3e5-d3feac72576b-kube-api-access-9gttx") pod "network-check-target-nq29s" (UID: "1598d282-cf11-48e5-b3e5-d3feac72576b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 11:47:52.756208 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:52.756171 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x982g" Sep 11 11:47:52.756823 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:52.756339 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x982g" podUID="d4fd9da2-13c7-4b85-9c89-f10a28d43049" Sep 11 11:47:52.756823 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:52.756425 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nq29s" Sep 11 11:47:52.756823 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:52.756495 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nq29s" podUID="1598d282-cf11-48e5-b3e5-d3feac72576b" Sep 11 11:47:52.756823 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:52.756553 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfx78" Sep 11 11:47:52.756823 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:52.756632 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfx78" podUID="2dc78d6d-cf10-4087-a38b-0ded1057f222" Sep 11 11:47:52.781602 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:52.780889 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-170.ec2.internal" event={"ID":"579298bbec0b9e8e74fae2628d563376","Type":"ContainerStarted","Data":"87a350349e2276a7e24e1e0cafb01a0cd25b71130c068061500ea22493622d2d"} Sep 11 11:47:52.796926 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:52.796810 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-170.ec2.internal" podStartSLOduration=3.796793476 podStartE2EDuration="3.796793476s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:47:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:47:52.79629251 +0000 UTC m=+4.561265688" watchObservedRunningTime="2026-09-11 11:47:52.796793476 +0000 UTC m=+4.561766652" Sep 11 11:47:53.793712 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:53.793627 2566 generic.go:356] "Generic (PLEG): container finished" podID="bf7554308d9b300ebecd820a92d9c922" containerID="2084d2914273970296d9e86373e76981ce56b508fa13a8e00e176c15918f540b" exitCode=0 Sep 11 11:47:53.794652 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:53.794605 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-170.ec2.internal" event={"ID":"bf7554308d9b300ebecd820a92d9c922","Type":"ContainerDied","Data":"2084d2914273970296d9e86373e76981ce56b508fa13a8e00e176c15918f540b"} Sep 11 11:47:54.298283 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:54.297769 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2dc78d6d-cf10-4087-a38b-0ded1057f222-original-pull-secret\") pod \"global-pull-secret-syncer-tfx78\" (UID: \"2dc78d6d-cf10-4087-a38b-0ded1057f222\") " pod="kube-system/global-pull-secret-syncer-tfx78" Sep 11 11:47:54.298283 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:54.297858 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4fd9da2-13c7-4b85-9c89-f10a28d43049-metrics-certs\") pod \"network-metrics-daemon-x982g\" (UID: \"d4fd9da2-13c7-4b85-9c89-f10a28d43049\") " pod="openshift-multus/network-metrics-daemon-x982g" Sep 11 11:47:54.298283 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:54.297969 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 11:47:54.298283 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:54.298036 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4fd9da2-13c7-4b85-9c89-f10a28d43049-metrics-certs podName:d4fd9da2-13c7-4b85-9c89-f10a28d43049 nodeName:}" failed. No retries permitted until 2026-09-11 11:47:58.298017567 +0000 UTC m=+10.062990735 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4fd9da2-13c7-4b85-9c89-f10a28d43049-metrics-certs") pod "network-metrics-daemon-x982g" (UID: "d4fd9da2-13c7-4b85-9c89-f10a28d43049") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 11:47:54.298283 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:54.297969 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 11 11:47:54.298283 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:54.298175 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2dc78d6d-cf10-4087-a38b-0ded1057f222-original-pull-secret podName:2dc78d6d-cf10-4087-a38b-0ded1057f222 nodeName:}" failed. No retries permitted until 2026-09-11 11:47:58.298161151 +0000 UTC m=+10.063134318 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2dc78d6d-cf10-4087-a38b-0ded1057f222-original-pull-secret") pod "global-pull-secret-syncer-tfx78" (UID: "2dc78d6d-cf10-4087-a38b-0ded1057f222") : object "kube-system"/"original-pull-secret" not registered Sep 11 11:47:54.399183 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:54.399136 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gttx\" (UniqueName: \"kubernetes.io/projected/1598d282-cf11-48e5-b3e5-d3feac72576b-kube-api-access-9gttx\") pod \"network-check-target-nq29s\" (UID: \"1598d282-cf11-48e5-b3e5-d3feac72576b\") " pod="openshift-network-diagnostics/network-check-target-nq29s" Sep 11 11:47:54.399355 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:54.399304 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 11 11:47:54.399355 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:54.399328 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 11 11:47:54.399355 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:54.399341 2566 projected.go:196] Error preparing data for projected volume kube-api-access-9gttx for pod openshift-network-diagnostics/network-check-target-nq29s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 11:47:54.399538 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:54.399443 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1598d282-cf11-48e5-b3e5-d3feac72576b-kube-api-access-9gttx podName:1598d282-cf11-48e5-b3e5-d3feac72576b nodeName:}" failed. No retries permitted until 2026-09-11 11:47:58.399422779 +0000 UTC m=+10.164395938 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-9gttx" (UniqueName: "kubernetes.io/projected/1598d282-cf11-48e5-b3e5-d3feac72576b-kube-api-access-9gttx") pod "network-check-target-nq29s" (UID: "1598d282-cf11-48e5-b3e5-d3feac72576b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 11:47:54.752246 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:54.752197 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfx78" Sep 11 11:47:54.752424 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:54.752329 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfx78" podUID="2dc78d6d-cf10-4087-a38b-0ded1057f222" Sep 11 11:47:54.752735 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:54.752714 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x982g" Sep 11 11:47:54.752843 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:54.752824 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x982g" podUID="d4fd9da2-13c7-4b85-9c89-f10a28d43049" Sep 11 11:47:54.752922 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:54.752909 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nq29s" Sep 11 11:47:54.753019 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:54.752995 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nq29s" podUID="1598d282-cf11-48e5-b3e5-d3feac72576b" Sep 11 11:47:56.755883 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:56.755069 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x982g" Sep 11 11:47:56.755883 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:56.755206 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x982g" podUID="d4fd9da2-13c7-4b85-9c89-f10a28d43049" Sep 11 11:47:56.755883 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:56.755570 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nq29s" Sep 11 11:47:56.755883 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:56.755657 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nq29s" podUID="1598d282-cf11-48e5-b3e5-d3feac72576b" Sep 11 11:47:56.755883 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:56.755711 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfx78" Sep 11 11:47:56.755883 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:56.755776 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfx78" podUID="2dc78d6d-cf10-4087-a38b-0ded1057f222" Sep 11 11:47:58.335464 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:58.335209 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2dc78d6d-cf10-4087-a38b-0ded1057f222-original-pull-secret\") pod \"global-pull-secret-syncer-tfx78\" (UID: \"2dc78d6d-cf10-4087-a38b-0ded1057f222\") " pod="kube-system/global-pull-secret-syncer-tfx78" Sep 11 11:47:58.335464 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:58.335295 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4fd9da2-13c7-4b85-9c89-f10a28d43049-metrics-certs\") pod \"network-metrics-daemon-x982g\" (UID: \"d4fd9da2-13c7-4b85-9c89-f10a28d43049\") " pod="openshift-multus/network-metrics-daemon-x982g" Sep 11 11:47:58.335464 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:58.335378 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 11 11:47:58.335464 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:58.335409 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 11:47:58.335464 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:58.335452 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2dc78d6d-cf10-4087-a38b-0ded1057f222-original-pull-secret podName:2dc78d6d-cf10-4087-a38b-0ded1057f222 nodeName:}" failed. No retries permitted until 2026-09-11 11:48:06.335432497 +0000 UTC m=+18.100405657 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2dc78d6d-cf10-4087-a38b-0ded1057f222-original-pull-secret") pod "global-pull-secret-syncer-tfx78" (UID: "2dc78d6d-cf10-4087-a38b-0ded1057f222") : object "kube-system"/"original-pull-secret" not registered Sep 11 11:47:58.335464 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:58.335474 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4fd9da2-13c7-4b85-9c89-f10a28d43049-metrics-certs podName:d4fd9da2-13c7-4b85-9c89-f10a28d43049 nodeName:}" failed. No retries permitted until 2026-09-11 11:48:06.335463988 +0000 UTC m=+18.100437146 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4fd9da2-13c7-4b85-9c89-f10a28d43049-metrics-certs") pod "network-metrics-daemon-x982g" (UID: "d4fd9da2-13c7-4b85-9c89-f10a28d43049") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 11:47:58.435883 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:58.435834 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gttx\" (UniqueName: \"kubernetes.io/projected/1598d282-cf11-48e5-b3e5-d3feac72576b-kube-api-access-9gttx\") pod \"network-check-target-nq29s\" (UID: \"1598d282-cf11-48e5-b3e5-d3feac72576b\") " pod="openshift-network-diagnostics/network-check-target-nq29s" Sep 11 11:47:58.436092 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:58.436070 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 11 11:47:58.436092 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:58.436093 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 11 11:47:58.436274 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:58.436105 2566 projected.go:196] Error preparing data for projected volume kube-api-access-9gttx for pod openshift-network-diagnostics/network-check-target-nq29s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 11:47:58.436274 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:58.436166 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1598d282-cf11-48e5-b3e5-d3feac72576b-kube-api-access-9gttx podName:1598d282-cf11-48e5-b3e5-d3feac72576b nodeName:}" failed. No retries permitted until 2026-09-11 11:48:06.436147827 +0000 UTC m=+18.201120995 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-9gttx" (UniqueName: "kubernetes.io/projected/1598d282-cf11-48e5-b3e5-d3feac72576b-kube-api-access-9gttx") pod "network-check-target-nq29s" (UID: "1598d282-cf11-48e5-b3e5-d3feac72576b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 11:47:58.753043 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:58.752965 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfx78" Sep 11 11:47:58.753206 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:58.753096 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfx78" podUID="2dc78d6d-cf10-4087-a38b-0ded1057f222" Sep 11 11:47:58.754620 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:58.753461 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x982g" Sep 11 11:47:58.754620 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:58.753570 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x982g" podUID="d4fd9da2-13c7-4b85-9c89-f10a28d43049" Sep 11 11:47:58.754992 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:47:58.754837 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nq29s" Sep 11 11:47:58.754992 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:47:58.754952 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nq29s" podUID="1598d282-cf11-48e5-b3e5-d3feac72576b" Sep 11 11:48:00.752638 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:00.752603 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nq29s" Sep 11 11:48:00.753160 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:00.752603 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x982g" Sep 11 11:48:00.753160 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:00.752728 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nq29s" podUID="1598d282-cf11-48e5-b3e5-d3feac72576b" Sep 11 11:48:00.753160 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:00.752743 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfx78" Sep 11 11:48:00.753160 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:00.752841 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x982g" podUID="d4fd9da2-13c7-4b85-9c89-f10a28d43049" Sep 11 11:48:00.753160 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:00.752991 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfx78" podUID="2dc78d6d-cf10-4087-a38b-0ded1057f222" Sep 11 11:48:02.753113 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:02.753078 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nq29s" Sep 11 11:48:02.753113 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:02.753113 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfx78" Sep 11 11:48:02.753653 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:02.753078 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x982g" Sep 11 11:48:02.753653 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:02.753196 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nq29s" podUID="1598d282-cf11-48e5-b3e5-d3feac72576b" Sep 11 11:48:02.753653 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:02.753285 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfx78" podUID="2dc78d6d-cf10-4087-a38b-0ded1057f222" Sep 11 11:48:02.753653 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:02.753365 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x982g" podUID="d4fd9da2-13c7-4b85-9c89-f10a28d43049" Sep 11 11:48:04.753243 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:04.753206 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x982g" Sep 11 11:48:04.753677 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:04.753206 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfx78" Sep 11 11:48:04.753677 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:04.753361 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x982g" podUID="d4fd9da2-13c7-4b85-9c89-f10a28d43049" Sep 11 11:48:04.753677 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:04.753206 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nq29s" Sep 11 11:48:04.753677 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:04.753429 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfx78" podUID="2dc78d6d-cf10-4087-a38b-0ded1057f222" Sep 11 11:48:04.753677 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:04.753475 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nq29s" podUID="1598d282-cf11-48e5-b3e5-d3feac72576b" Sep 11 11:48:06.390449 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:06.390416 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2dc78d6d-cf10-4087-a38b-0ded1057f222-original-pull-secret\") pod \"global-pull-secret-syncer-tfx78\" (UID: \"2dc78d6d-cf10-4087-a38b-0ded1057f222\") " pod="kube-system/global-pull-secret-syncer-tfx78" Sep 11 11:48:06.390922 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:06.390504 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4fd9da2-13c7-4b85-9c89-f10a28d43049-metrics-certs\") pod \"network-metrics-daemon-x982g\" (UID: \"d4fd9da2-13c7-4b85-9c89-f10a28d43049\") " pod="openshift-multus/network-metrics-daemon-x982g" Sep 11 11:48:06.390922 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:06.390594 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 11 11:48:06.390922 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:06.390673 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2dc78d6d-cf10-4087-a38b-0ded1057f222-original-pull-secret podName:2dc78d6d-cf10-4087-a38b-0ded1057f222 nodeName:}" failed. No retries permitted until 2026-09-11 11:48:22.390652151 +0000 UTC m=+34.155625308 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2dc78d6d-cf10-4087-a38b-0ded1057f222-original-pull-secret") pod "global-pull-secret-syncer-tfx78" (UID: "2dc78d6d-cf10-4087-a38b-0ded1057f222") : object "kube-system"/"original-pull-secret" not registered Sep 11 11:48:06.390922 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:06.390598 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 11:48:06.390922 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:06.390737 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4fd9da2-13c7-4b85-9c89-f10a28d43049-metrics-certs podName:d4fd9da2-13c7-4b85-9c89-f10a28d43049 nodeName:}" failed. No retries permitted until 2026-09-11 11:48:22.390721049 +0000 UTC m=+34.155694203 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4fd9da2-13c7-4b85-9c89-f10a28d43049-metrics-certs") pod "network-metrics-daemon-x982g" (UID: "d4fd9da2-13c7-4b85-9c89-f10a28d43049") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 11:48:06.491085 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:06.491043 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gttx\" (UniqueName: \"kubernetes.io/projected/1598d282-cf11-48e5-b3e5-d3feac72576b-kube-api-access-9gttx\") pod \"network-check-target-nq29s\" (UID: \"1598d282-cf11-48e5-b3e5-d3feac72576b\") " pod="openshift-network-diagnostics/network-check-target-nq29s" Sep 11 11:48:06.491250 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:06.491185 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 11 11:48:06.491250 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:06.491202 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 11 11:48:06.491250 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:06.491211 2566 projected.go:196] Error preparing data for projected volume kube-api-access-9gttx for pod openshift-network-diagnostics/network-check-target-nq29s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 11:48:06.491419 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:06.491267 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1598d282-cf11-48e5-b3e5-d3feac72576b-kube-api-access-9gttx podName:1598d282-cf11-48e5-b3e5-d3feac72576b nodeName:}" failed. No retries permitted until 2026-09-11 11:48:22.491249796 +0000 UTC m=+34.256222951 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-9gttx" (UniqueName: "kubernetes.io/projected/1598d282-cf11-48e5-b3e5-d3feac72576b-kube-api-access-9gttx") pod "network-check-target-nq29s" (UID: "1598d282-cf11-48e5-b3e5-d3feac72576b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 11:48:06.752614 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:06.752532 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x982g" Sep 11 11:48:06.752614 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:06.752570 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfx78" Sep 11 11:48:06.752822 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:06.752532 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nq29s" Sep 11 11:48:06.752822 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:06.752664 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x982g" podUID="d4fd9da2-13c7-4b85-9c89-f10a28d43049" Sep 11 11:48:06.752822 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:06.752753 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nq29s" podUID="1598d282-cf11-48e5-b3e5-d3feac72576b" Sep 11 11:48:06.752822 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:06.752813 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfx78" podUID="2dc78d6d-cf10-4087-a38b-0ded1057f222" Sep 11 11:48:08.753237 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:08.753203 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfx78" Sep 11 11:48:08.753630 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:08.753285 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfx78" podUID="2dc78d6d-cf10-4087-a38b-0ded1057f222" Sep 11 11:48:08.753630 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:08.753329 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nq29s" Sep 11 11:48:08.753630 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:08.753367 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nq29s" podUID="1598d282-cf11-48e5-b3e5-d3feac72576b" Sep 11 11:48:08.753630 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:08.753397 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x982g" Sep 11 11:48:08.753630 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:08.753443 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x982g" podUID="d4fd9da2-13c7-4b85-9c89-f10a28d43049" Sep 11 11:48:09.842074 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:09.841835 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hc6pd" event={"ID":"2d1abaa8-2eb8-4feb-9452-a55924a95452","Type":"ContainerStarted","Data":"c831ea59a32924977f1425d28a84ece6b0a34462696cd5b10df698b786cbce72"} Sep 11 11:48:09.843199 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:09.843175 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2sjcg" event={"ID":"e0202693-9deb-4f3b-8d1f-3eec2f3f03d6","Type":"ContainerStarted","Data":"70cca891671e4f7cc75480debbb159c3e6eaf0d2a7af516b1d168d2148ac43b7"} Sep 11 11:48:09.844452 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:09.844423 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" event={"ID":"e7a5ff3b-7309-483c-bc1f-3b0d86c2216e","Type":"ContainerStarted","Data":"c154cf0d278b1cf685d9228d4b48e4d88ceb216152596cb080fe406482701a11"} Sep 11 11:48:09.845718 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:09.845697 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bf99t" event={"ID":"1f35e6ed-521d-436d-8bf6-ee60e8d8fa05","Type":"ContainerStarted","Data":"b8e4fc93eba2778caf5be6e5f339a6d537b309810bcf80ebe7b609995f70ccf6"} Sep 11 11:48:09.847100 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:09.847076 2566 generic.go:356] "Generic (PLEG): container finished" podID="091f9985-8446-4191-9ec0-9f40b1d1a5fb" containerID="3d3a98dad766d02d92dd7d20092f6e4269c41a42af35a8275938e5b22b948e77" exitCode=0 Sep 11 11:48:09.847188 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:09.847165 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cs8ww" event={"ID":"091f9985-8446-4191-9ec0-9f40b1d1a5fb","Type":"ContainerDied","Data":"3d3a98dad766d02d92dd7d20092f6e4269c41a42af35a8275938e5b22b948e77"} Sep 11 11:48:09.848996 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:09.848973 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-170.ec2.internal" event={"ID":"bf7554308d9b300ebecd820a92d9c922","Type":"ContainerStarted","Data":"5afd92ef7d9b3a87a09bf980764fbd16e80e1ecac232788a2282fec860604f14"} Sep 11 11:48:09.851982 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:09.851952 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" event={"ID":"c48ecd03-9f9f-4c11-95f3-b7400599b444","Type":"ContainerStarted","Data":"a3b455addfa371e3ba8d9ff0734667faf94650963f03d1989dce5e034e73ca04"} Sep 11 11:48:09.852092 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:09.852075 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" event={"ID":"c48ecd03-9f9f-4c11-95f3-b7400599b444","Type":"ContainerStarted","Data":"6f69512badfda32d46596e7434b11d593719d1922e2a71b3a0a604bd3c3a7782"} Sep 11 11:48:09.852155 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:09.852133 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" event={"ID":"c48ecd03-9f9f-4c11-95f3-b7400599b444","Type":"ContainerStarted","Data":"2c1e2a645cab4a53882a1a4841667b1a30fb87c00eac74a1b242d76deebffb34"} Sep 11 11:48:09.852234 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:09.852221 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" event={"ID":"c48ecd03-9f9f-4c11-95f3-b7400599b444","Type":"ContainerStarted","Data":"a5d347f8328677ed405a33637668d0c39ee72d783328c2f9390c1e77ebb71d8c"} Sep 11 11:48:09.852286 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:09.852273 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" event={"ID":"c48ecd03-9f9f-4c11-95f3-b7400599b444","Type":"ContainerStarted","Data":"94503bd8bb344857a84961fa49cdc9a8d7f786801d62368c825758c905bc3602"} Sep 11 11:48:09.852334 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:09.852321 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" event={"ID":"c48ecd03-9f9f-4c11-95f3-b7400599b444","Type":"ContainerStarted","Data":"29d77dd51948b1f76506e67424c57e9bba255e64ed76c8f38d710a0c26bb06cb"} Sep 11 11:48:09.853224 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:09.853205 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-n9pc8" event={"ID":"f90862ea-2909-4701-ab56-fe2d3fbb83f8","Type":"ContainerStarted","Data":"3cd3945e3ace214af786901c89bfaf8049a5c0e49ab8a3381cba4b42533c7fbb"} Sep 11 11:48:09.854890 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:09.854856 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rz4hn" event={"ID":"5bf5658e-69eb-4ac8-9359-4653f8fd588f","Type":"ContainerStarted","Data":"d75aaa386fc14c737c63a038b197b2786d796f78e76cf31bfa627f5f275009c7"} Sep 11 11:48:09.870877 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:09.870832 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-2sjcg" podStartSLOduration=3.983029453 podStartE2EDuration="21.870819875s" totalImagesPullingTime="17.887790422s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:47:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:48:09.870796208 +0000 UTC m=+21.635769382" watchObservedRunningTime="2026-09-11 11:48:09.870819875 +0000 UTC m=+21.635793102" Sep 11 11:48:09.886197 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:09.886126 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-n9pc8" podStartSLOduration=6.997613043 podStartE2EDuration="20.886113262s" totalImagesPullingTime="13.888500219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:47:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:48:09.885849478 +0000 UTC m=+21.650822653" watchObservedRunningTime="2026-09-11 11:48:09.886113262 +0000 UTC m=+21.651086436" Sep 11 11:48:09.905861 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:09.905825 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-bf99t" podStartSLOduration=3.028943515 podStartE2EDuration="20.90581218s" totalImagesPullingTime="17.876868665s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:47:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:48:09.905319096 +0000 UTC m=+21.670292270" watchObservedRunningTime="2026-09-11 11:48:09.90581218 +0000 UTC m=+21.670785375" Sep 11 11:48:09.928447 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:09.928411 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fm2g2" podStartSLOduration=3.079679881 podStartE2EDuration="20.928399057s" totalImagesPullingTime="17.848719176s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:47:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:48:09.928244542 +0000 UTC m=+21.693217716" watchObservedRunningTime="2026-09-11 11:48:09.928399057 +0000 UTC m=+21.693372232" Sep 11 11:48:09.950687 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:09.950649 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-170.ec2.internal" podStartSLOduration=20.950638649 podStartE2EDuration="20.950638649s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:47:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:48:09.950169478 +0000 UTC m=+21.715142652" watchObservedRunningTime="2026-09-11 11:48:09.950638649 +0000 UTC m=+21.715611824" Sep 11 11:48:10.552821 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:10.552799 2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 11 11:48:10.719206 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:10.719057 2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-11T11:48:10.55281596Z","UUID":"4c25bfa6-320b-445a-af02-8dfa9716852f","Handler":null,"Name":"","Endpoint":""} Sep 11 11:48:10.720536 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:10.720517 2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 11 11:48:10.720536 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:10.720554 2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 11 11:48:10.752437 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:10.752408 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfx78" Sep 11 11:48:10.752437 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:10.752423 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nq29s" Sep 11 11:48:10.752635 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:10.752412 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x982g" Sep 11 11:48:10.752635 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:10.752536 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfx78" podUID="2dc78d6d-cf10-4087-a38b-0ded1057f222" Sep 11 11:48:10.752635 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:10.752623 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nq29s" podUID="1598d282-cf11-48e5-b3e5-d3feac72576b" Sep 11 11:48:10.752783 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:10.752723 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x982g" podUID="d4fd9da2-13c7-4b85-9c89-f10a28d43049" Sep 11 11:48:10.858742 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:10.858704 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hc6pd" event={"ID":"2d1abaa8-2eb8-4feb-9452-a55924a95452","Type":"ContainerStarted","Data":"d13589f5febeab621eea3d3f2d737871ccc2a1aff2aa02e0babc646a7f7b2688"} Sep 11 11:48:10.860219 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:10.860181 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-x6tjp" event={"ID":"990fcaba-28cc-458a-a2c6-8aaff01fd3ab","Type":"ContainerStarted","Data":"284819918a9689bf1ab1d3e12a78ac05903aedd4337dc92104478ef025e138df"} Sep 11 11:48:10.876500 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:10.876095 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-x6tjp" podStartSLOduration=5.2681495179999995 podStartE2EDuration="22.87607574s" totalImagesPullingTime="17.607926222s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:47:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:48:10.876064269 +0000 UTC m=+22.641037443" watchObservedRunningTime="2026-09-11 11:48:10.87607574 +0000 UTC m=+22.641048916" Sep 11 11:48:10.876728 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:10.876697 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rz4hn" podStartSLOduration=4.295750849 podStartE2EDuration="21.8766819s" totalImagesPullingTime="17.580931051s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:47:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:48:10.015519119 +0000 UTC m=+21.780492293" watchObservedRunningTime="2026-09-11 11:48:10.8766819 +0000 UTC m=+22.641655078" Sep 11 11:48:11.867223 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:11.867125 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" event={"ID":"c48ecd03-9f9f-4c11-95f3-b7400599b444","Type":"ContainerStarted","Data":"63389d9dae09d06ba4db80abd4af0fe2674cc9a8d5879ce57b8a705908e5baba"} Sep 11 11:48:11.869654 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:11.869620 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hc6pd" event={"ID":"2d1abaa8-2eb8-4feb-9452-a55924a95452","Type":"ContainerStarted","Data":"1082a22ccc0d9c12515b961a1525f5386436f322e6d9758bff289a8911c7a7eb"} Sep 11 11:48:12.752752 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:12.752715 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfx78" Sep 11 11:48:12.753147 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:12.752730 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x982g" Sep 11 11:48:12.753147 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:12.752844 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfx78" podUID="2dc78d6d-cf10-4087-a38b-0ded1057f222" Sep 11 11:48:12.753147 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:12.752855 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nq29s" Sep 11 11:48:12.753147 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:12.752919 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x982g" podUID="d4fd9da2-13c7-4b85-9c89-f10a28d43049" Sep 11 11:48:12.753147 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:12.752978 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nq29s" podUID="1598d282-cf11-48e5-b3e5-d3feac72576b" Sep 11 11:48:14.596883 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:14.596839 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-n9pc8" Sep 11 11:48:14.597533 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:14.597503 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-n9pc8" Sep 11 11:48:14.616725 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:14.616683 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hc6pd" podStartSLOduration=6.368298409 podStartE2EDuration="26.616668715s" totalImagesPullingTime="20.248370306s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:47:48 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:48:11.900264197 +0000 UTC m=+23.665237372" watchObservedRunningTime="2026-09-11 11:48:14.616668715 +0000 UTC m=+26.381641890" Sep 11 11:48:14.752994 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:14.752959 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nq29s" Sep 11 11:48:14.753167 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:14.752959 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x982g" Sep 11 11:48:14.753167 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:14.753077 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfx78" Sep 11 11:48:14.753167 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:14.753081 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nq29s" podUID="1598d282-cf11-48e5-b3e5-d3feac72576b" Sep 11 11:48:14.753167 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:14.753163 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x982g" podUID="d4fd9da2-13c7-4b85-9c89-f10a28d43049" Sep 11 11:48:14.753330 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:14.753268 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfx78" podUID="2dc78d6d-cf10-4087-a38b-0ded1057f222" Sep 11 11:48:15.879682 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:15.879506 2566 generic.go:356] "Generic (PLEG): container finished" podID="091f9985-8446-4191-9ec0-9f40b1d1a5fb" containerID="043a4385429563fa908f64370175802b4916ee099859fdedab2b79f7e949d169" exitCode=0 Sep 11 11:48:15.880105 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:15.879567 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cs8ww" event={"ID":"091f9985-8446-4191-9ec0-9f40b1d1a5fb","Type":"ContainerDied","Data":"043a4385429563fa908f64370175802b4916ee099859fdedab2b79f7e949d169"} Sep 11 11:48:15.883570 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:15.883547 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" event={"ID":"c48ecd03-9f9f-4c11-95f3-b7400599b444","Type":"ContainerStarted","Data":"6a4c1615eb94c27d925d1bf8b39300aaae045e497e76a2a1ceddff20bfd8b223"} Sep 11 11:48:15.883854 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:15.883836 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:48:15.898640 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:15.898614 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:48:16.163667 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:16.163577 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-n9pc8" Sep 11 11:48:16.163852 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:16.163683 2566 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 11 11:48:16.164133 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:16.164118 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-n9pc8" Sep 11 11:48:16.185311 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:16.185272 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" podStartSLOduration=8.81244244 podStartE2EDuration="27.185257505s" totalImagesPullingTime="18.372815065s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:47:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:48:15.934879605 +0000 UTC m=+27.699852777" watchObservedRunningTime="2026-09-11 11:48:16.185257505 +0000 UTC m=+27.950230679" Sep 11 11:48:16.752939 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:16.752909 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nq29s" Sep 11 11:48:16.753083 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:16.752909 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x982g" Sep 11 11:48:16.753083 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:16.753041 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nq29s" podUID="1598d282-cf11-48e5-b3e5-d3feac72576b" Sep 11 11:48:16.753203 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:16.753150 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x982g" podUID="d4fd9da2-13c7-4b85-9c89-f10a28d43049" Sep 11 11:48:16.753203 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:16.752913 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfx78" Sep 11 11:48:16.753298 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:16.753283 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfx78" podUID="2dc78d6d-cf10-4087-a38b-0ded1057f222" Sep 11 11:48:16.888653 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:16.888611 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cs8ww" event={"ID":"091f9985-8446-4191-9ec0-9f40b1d1a5fb","Type":"ContainerStarted","Data":"dbd08fe5bf6c923f715fa4b12fa1c87cdcebac6393c65d45f4e53894aeea25b7"} Sep 11 11:48:16.889250 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:16.889216 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:48:16.889312 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:16.889276 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:48:16.910378 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:16.910355 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:48:17.169458 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:17.169417 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nq29s"] Sep 11 11:48:17.169614 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:17.169531 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nq29s" Sep 11 11:48:17.169658 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:17.169607 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nq29s" podUID="1598d282-cf11-48e5-b3e5-d3feac72576b" Sep 11 11:48:17.183918 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:17.183886 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-x982g"] Sep 11 11:48:17.184045 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:17.184017 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x982g" Sep 11 11:48:17.184152 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:17.184132 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x982g" podUID="d4fd9da2-13c7-4b85-9c89-f10a28d43049" Sep 11 11:48:17.201861 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:17.201829 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tfx78"] Sep 11 11:48:17.202014 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:17.201960 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfx78" Sep 11 11:48:17.202141 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:17.202067 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfx78" podUID="2dc78d6d-cf10-4087-a38b-0ded1057f222" Sep 11 11:48:17.893687 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:17.893657 2566 generic.go:356] "Generic (PLEG): container finished" podID="091f9985-8446-4191-9ec0-9f40b1d1a5fb" containerID="dbd08fe5bf6c923f715fa4b12fa1c87cdcebac6393c65d45f4e53894aeea25b7" exitCode=0 Sep 11 11:48:17.894051 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:17.893740 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cs8ww" event={"ID":"091f9985-8446-4191-9ec0-9f40b1d1a5fb","Type":"ContainerDied","Data":"dbd08fe5bf6c923f715fa4b12fa1c87cdcebac6393c65d45f4e53894aeea25b7"} Sep 11 11:48:18.754345 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:18.754271 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nq29s" Sep 11 11:48:18.754483 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:18.754363 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfx78" Sep 11 11:48:18.754483 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:18.754403 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nq29s" podUID="1598d282-cf11-48e5-b3e5-d3feac72576b" Sep 11 11:48:18.754483 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:18.754442 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfx78" podUID="2dc78d6d-cf10-4087-a38b-0ded1057f222" Sep 11 11:48:18.754483 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:18.754469 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x982g" Sep 11 11:48:18.754618 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:18.754552 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x982g" podUID="d4fd9da2-13c7-4b85-9c89-f10a28d43049" Sep 11 11:48:19.900089 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:19.899899 2566 generic.go:356] "Generic (PLEG): container finished" podID="091f9985-8446-4191-9ec0-9f40b1d1a5fb" containerID="e7570ce12c4dc5caac3499e6b53d396df3ec95a207c1b1d828b080979abed733" exitCode=0 Sep 11 11:48:19.900583 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:19.899973 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cs8ww" event={"ID":"091f9985-8446-4191-9ec0-9f40b1d1a5fb","Type":"ContainerDied","Data":"e7570ce12c4dc5caac3499e6b53d396df3ec95a207c1b1d828b080979abed733"} Sep 11 11:48:20.753201 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:20.753152 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfx78" Sep 11 11:48:20.753415 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:20.753156 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x982g" Sep 11 11:48:20.753415 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:20.753160 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nq29s" Sep 11 11:48:20.753415 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:20.753305 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfx78" podUID="2dc78d6d-cf10-4087-a38b-0ded1057f222" Sep 11 11:48:20.753570 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:20.753429 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nq29s" podUID="1598d282-cf11-48e5-b3e5-d3feac72576b" Sep 11 11:48:20.753570 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:20.753531 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x982g" podUID="d4fd9da2-13c7-4b85-9c89-f10a28d43049" Sep 11 11:48:22.080144 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.080114 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-170.ec2.internal" event="NodeReady" Sep 11 11:48:22.080512 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.080248 2566 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 11 11:48:22.146149 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.146023 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-9f4c766d4-g58wm"] Sep 11 11:48:22.161220 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.161189 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" Sep 11 11:48:22.164663 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.164588 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 11 11:48:22.164790 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.164755 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-47c8b\"" Sep 11 11:48:22.165719 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.165693 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 11 11:48:22.165836 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.165736 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 11 11:48:22.170642 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.170281 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9f582"] Sep 11 11:48:22.175184 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.175159 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 11 11:48:22.181650 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.181629 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5qrzc"] Sep 11 11:48:22.181754 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.181632 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9f582" Sep 11 11:48:22.184574 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.184554 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 11 11:48:22.184873 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.184854 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-m9sw5\"" Sep 11 11:48:22.185058 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.185042 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 11 11:48:22.193845 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.193824 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-9f4c766d4-g58wm"] Sep 11 11:48:22.193972 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.193860 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5qrzc"] Sep 11 11:48:22.193972 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.193965 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5qrzc" Sep 11 11:48:22.194377 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.194137 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9f582"] Sep 11 11:48:22.196291 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.196272 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 11 11:48:22.196388 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.196272 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wpl6b\"" Sep 11 11:48:22.196449 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.196275 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 11 11:48:22.196581 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.196564 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 11 11:48:22.304668 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.304633 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1190be0-add2-48ea-ad87-5d7271697bfa-cert\") pod \"ingress-canary-5qrzc\" (UID: \"d1190be0-add2-48ea-ad87-5d7271697bfa\") " pod="openshift-ingress-canary/ingress-canary-5qrzc" Sep 11 11:48:22.304668 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.304676 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlmpl\" (UniqueName: \"kubernetes.io/projected/d1190be0-add2-48ea-ad87-5d7271697bfa-kube-api-access-qlmpl\") pod \"ingress-canary-5qrzc\" (UID: \"d1190be0-add2-48ea-ad87-5d7271697bfa\") " pod="openshift-ingress-canary/ingress-canary-5qrzc" Sep 11 11:48:22.304953 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.304768 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/60eec8ae-8c59-40f6-820a-9671b9ec919c-ca-trust-extracted\") pod \"image-registry-9f4c766d4-g58wm\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" Sep 11 11:48:22.304953 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.304808 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/60eec8ae-8c59-40f6-820a-9671b9ec919c-installation-pull-secrets\") pod \"image-registry-9f4c766d4-g58wm\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" Sep 11 11:48:22.304953 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.304843 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-bound-sa-token\") pod \"image-registry-9f4c766d4-g58wm\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" Sep 11 11:48:22.304953 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.304889 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nknpl\" (UniqueName: \"kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-kube-api-access-nknpl\") pod \"image-registry-9f4c766d4-g58wm\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" Sep 11 11:48:22.304953 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.304916 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6673d6af-e1b2-443b-83c2-7dc7b653b9d0-config-volume\") pod \"dns-default-9f582\" (UID: \"6673d6af-e1b2-443b-83c2-7dc7b653b9d0\") " pod="openshift-dns/dns-default-9f582" Sep 11 11:48:22.304953 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.304935 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6673d6af-e1b2-443b-83c2-7dc7b653b9d0-metrics-tls\") pod \"dns-default-9f582\" (UID: \"6673d6af-e1b2-443b-83c2-7dc7b653b9d0\") " pod="openshift-dns/dns-default-9f582" Sep 11 11:48:22.305193 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.304973 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/60eec8ae-8c59-40f6-820a-9671b9ec919c-trusted-ca\") pod \"image-registry-9f4c766d4-g58wm\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" Sep 11 11:48:22.305193 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.304988 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6673d6af-e1b2-443b-83c2-7dc7b653b9d0-tmp-dir\") pod \"dns-default-9f582\" (UID: \"6673d6af-e1b2-443b-83c2-7dc7b653b9d0\") " pod="openshift-dns/dns-default-9f582" Sep 11 11:48:22.305193 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.305040 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tjc4\" (UniqueName: \"kubernetes.io/projected/6673d6af-e1b2-443b-83c2-7dc7b653b9d0-kube-api-access-9tjc4\") pod \"dns-default-9f582\" (UID: \"6673d6af-e1b2-443b-83c2-7dc7b653b9d0\") " pod="openshift-dns/dns-default-9f582" Sep 11 11:48:22.305193 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.305059 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-registry-tls\") pod \"image-registry-9f4c766d4-g58wm\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" Sep 11 11:48:22.305193 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.305073 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/60eec8ae-8c59-40f6-820a-9671b9ec919c-registry-certificates\") pod \"image-registry-9f4c766d4-g58wm\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" Sep 11 11:48:22.305193 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.305101 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/60eec8ae-8c59-40f6-820a-9671b9ec919c-image-registry-private-configuration\") pod \"image-registry-9f4c766d4-g58wm\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" Sep 11 11:48:22.339850 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.339819 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n9pc8_f90862ea-2909-4701-ab56-fe2d3fbb83f8/konnectivity-agent/0.log" limit=1024 Sep 11 11:48:22.405939 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.405894 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/60eec8ae-8c59-40f6-820a-9671b9ec919c-image-registry-private-configuration\") pod \"image-registry-9f4c766d4-g58wm\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" Sep 11 11:48:22.405939 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.405943 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1190be0-add2-48ea-ad87-5d7271697bfa-cert\") pod \"ingress-canary-5qrzc\" (UID: \"d1190be0-add2-48ea-ad87-5d7271697bfa\") " pod="openshift-ingress-canary/ingress-canary-5qrzc" Sep 11 11:48:22.406198 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.405970 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlmpl\" (UniqueName: \"kubernetes.io/projected/d1190be0-add2-48ea-ad87-5d7271697bfa-kube-api-access-qlmpl\") pod \"ingress-canary-5qrzc\" (UID: \"d1190be0-add2-48ea-ad87-5d7271697bfa\") " pod="openshift-ingress-canary/ingress-canary-5qrzc" Sep 11 11:48:22.406198 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.406013 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2dc78d6d-cf10-4087-a38b-0ded1057f222-original-pull-secret\") pod \"global-pull-secret-syncer-tfx78\" (UID: \"2dc78d6d-cf10-4087-a38b-0ded1057f222\") " pod="kube-system/global-pull-secret-syncer-tfx78" Sep 11 11:48:22.406198 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.406056 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/60eec8ae-8c59-40f6-820a-9671b9ec919c-ca-trust-extracted\") pod \"image-registry-9f4c766d4-g58wm\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" Sep 11 11:48:22.406198 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.406081 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/60eec8ae-8c59-40f6-820a-9671b9ec919c-installation-pull-secrets\") pod \"image-registry-9f4c766d4-g58wm\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" Sep 11 11:48:22.406198 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.406106 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-bound-sa-token\") pod \"image-registry-9f4c766d4-g58wm\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" Sep 11 11:48:22.406198 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.406129 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nknpl\" (UniqueName: \"kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-kube-api-access-nknpl\") pod \"image-registry-9f4c766d4-g58wm\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" Sep 11 11:48:22.406198 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.406154 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6673d6af-e1b2-443b-83c2-7dc7b653b9d0-config-volume\") pod \"dns-default-9f582\" (UID: \"6673d6af-e1b2-443b-83c2-7dc7b653b9d0\") " pod="openshift-dns/dns-default-9f582" Sep 11 11:48:22.406198 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.406180 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6673d6af-e1b2-443b-83c2-7dc7b653b9d0-metrics-tls\") pod \"dns-default-9f582\" (UID: \"6673d6af-e1b2-443b-83c2-7dc7b653b9d0\") " pod="openshift-dns/dns-default-9f582" Sep 11 11:48:22.406560 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.406222 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/60eec8ae-8c59-40f6-820a-9671b9ec919c-trusted-ca\") pod \"image-registry-9f4c766d4-g58wm\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" Sep 11 11:48:22.406560 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.406242 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6673d6af-e1b2-443b-83c2-7dc7b653b9d0-tmp-dir\") pod \"dns-default-9f582\" (UID: \"6673d6af-e1b2-443b-83c2-7dc7b653b9d0\") " pod="openshift-dns/dns-default-9f582" Sep 11 11:48:22.406560 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.406267 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tjc4\" (UniqueName: \"kubernetes.io/projected/6673d6af-e1b2-443b-83c2-7dc7b653b9d0-kube-api-access-9tjc4\") pod \"dns-default-9f582\" (UID: \"6673d6af-e1b2-443b-83c2-7dc7b653b9d0\") " pod="openshift-dns/dns-default-9f582" Sep 11 11:48:22.406560 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.406289 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-registry-tls\") pod \"image-registry-9f4c766d4-g58wm\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" Sep 11 11:48:22.406560 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.406311 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/60eec8ae-8c59-40f6-820a-9671b9ec919c-registry-certificates\") pod \"image-registry-9f4c766d4-g58wm\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" Sep 11 11:48:22.406560 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.406332 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4fd9da2-13c7-4b85-9c89-f10a28d43049-metrics-certs\") pod \"network-metrics-daemon-x982g\" (UID: \"d4fd9da2-13c7-4b85-9c89-f10a28d43049\") " pod="openshift-multus/network-metrics-daemon-x982g" Sep 11 11:48:22.406560 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:22.406436 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 11:48:22.406560 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:22.406493 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4fd9da2-13c7-4b85-9c89-f10a28d43049-metrics-certs podName:d4fd9da2-13c7-4b85-9c89-f10a28d43049 nodeName:}" failed. No retries permitted until 2026-09-11 11:48:54.406476208 +0000 UTC m=+66.171449374 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4fd9da2-13c7-4b85-9c89-f10a28d43049-metrics-certs") pod "network-metrics-daemon-x982g" (UID: "d4fd9da2-13c7-4b85-9c89-f10a28d43049") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 11:48:22.406951 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:22.406669 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 11 11:48:22.406951 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:22.406725 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6673d6af-e1b2-443b-83c2-7dc7b653b9d0-metrics-tls podName:6673d6af-e1b2-443b-83c2-7dc7b653b9d0 nodeName:}" failed. No retries permitted until 2026-09-11 11:48:22.906707733 +0000 UTC m=+34.671680891 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6673d6af-e1b2-443b-83c2-7dc7b653b9d0-metrics-tls") pod "dns-default-9f582" (UID: "6673d6af-e1b2-443b-83c2-7dc7b653b9d0") : secret "dns-default-metrics-tls" not found Sep 11 11:48:22.407122 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:22.407037 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 11 11:48:22.407122 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:22.407057 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9f4c766d4-g58wm: secret "image-registry-tls" not found Sep 11 11:48:22.407122 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:22.407103 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-registry-tls podName:60eec8ae-8c59-40f6-820a-9671b9ec919c nodeName:}" failed. No retries permitted until 2026-09-11 11:48:22.907087494 +0000 UTC m=+34.672060654 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-registry-tls") pod "image-registry-9f4c766d4-g58wm" (UID: "60eec8ae-8c59-40f6-820a-9671b9ec919c") : secret "image-registry-tls" not found Sep 11 11:48:22.407410 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.407380 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6673d6af-e1b2-443b-83c2-7dc7b653b9d0-tmp-dir\") pod \"dns-default-9f582\" (UID: \"6673d6af-e1b2-443b-83c2-7dc7b653b9d0\") " pod="openshift-dns/dns-default-9f582" Sep 11 11:48:22.407533 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:22.407473 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 11 11:48:22.407533 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:22.407520 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2dc78d6d-cf10-4087-a38b-0ded1057f222-original-pull-secret podName:2dc78d6d-cf10-4087-a38b-0ded1057f222 nodeName:}" failed. No retries permitted until 2026-09-11 11:48:54.407505383 +0000 UTC m=+66.172478589 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2dc78d6d-cf10-4087-a38b-0ded1057f222-original-pull-secret") pod "global-pull-secret-syncer-tfx78" (UID: "2dc78d6d-cf10-4087-a38b-0ded1057f222") : object "kube-system"/"original-pull-secret" not registered Sep 11 11:48:22.407656 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.407553 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6673d6af-e1b2-443b-83c2-7dc7b653b9d0-config-volume\") pod \"dns-default-9f582\" (UID: \"6673d6af-e1b2-443b-83c2-7dc7b653b9d0\") " pod="openshift-dns/dns-default-9f582" Sep 11 11:48:22.407846 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.407813 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/60eec8ae-8c59-40f6-820a-9671b9ec919c-trusted-ca\") pod \"image-registry-9f4c766d4-g58wm\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" Sep 11 11:48:22.408343 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.408317 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/60eec8ae-8c59-40f6-820a-9671b9ec919c-registry-certificates\") pod \"image-registry-9f4c766d4-g58wm\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" Sep 11 11:48:22.408542 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.408514 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/60eec8ae-8c59-40f6-820a-9671b9ec919c-ca-trust-extracted\") pod \"image-registry-9f4c766d4-g58wm\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" Sep 11 11:48:22.410615 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.410561 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/60eec8ae-8c59-40f6-820a-9671b9ec919c-image-registry-private-configuration\") pod \"image-registry-9f4c766d4-g58wm\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" Sep 11 11:48:22.410615 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.410581 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/60eec8ae-8c59-40f6-820a-9671b9ec919c-installation-pull-secrets\") pod \"image-registry-9f4c766d4-g58wm\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" Sep 11 11:48:22.410615 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.410588 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1190be0-add2-48ea-ad87-5d7271697bfa-cert\") pod \"ingress-canary-5qrzc\" (UID: \"d1190be0-add2-48ea-ad87-5d7271697bfa\") " pod="openshift-ingress-canary/ingress-canary-5qrzc" Sep 11 11:48:22.416140 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.416061 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-bound-sa-token\") pod \"image-registry-9f4c766d4-g58wm\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" Sep 11 11:48:22.416238 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.416142 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nknpl\" (UniqueName: \"kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-kube-api-access-nknpl\") pod \"image-registry-9f4c766d4-g58wm\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" Sep 11 11:48:22.417134 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.417092 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tjc4\" (UniqueName: \"kubernetes.io/projected/6673d6af-e1b2-443b-83c2-7dc7b653b9d0-kube-api-access-9tjc4\") pod \"dns-default-9f582\" (UID: \"6673d6af-e1b2-443b-83c2-7dc7b653b9d0\") " pod="openshift-dns/dns-default-9f582" Sep 11 11:48:22.417796 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.417755 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlmpl\" (UniqueName: \"kubernetes.io/projected/d1190be0-add2-48ea-ad87-5d7271697bfa-kube-api-access-qlmpl\") pod \"ingress-canary-5qrzc\" (UID: \"d1190be0-add2-48ea-ad87-5d7271697bfa\") " pod="openshift-ingress-canary/ingress-canary-5qrzc" Sep 11 11:48:22.488375 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.488339 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n9pc8_f90862ea-2909-4701-ab56-fe2d3fbb83f8/konnectivity-agent/0.log" limit=1024 Sep 11 11:48:22.503801 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.503775 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5qrzc" Sep 11 11:48:22.507130 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.507106 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gttx\" (UniqueName: \"kubernetes.io/projected/1598d282-cf11-48e5-b3e5-d3feac72576b-kube-api-access-9gttx\") pod \"network-check-target-nq29s\" (UID: \"1598d282-cf11-48e5-b3e5-d3feac72576b\") " pod="openshift-network-diagnostics/network-check-target-nq29s" Sep 11 11:48:22.507309 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:22.507292 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 11 11:48:22.507366 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:22.507319 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 11 11:48:22.507366 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:22.507332 2566 projected.go:196] Error preparing data for projected volume kube-api-access-9gttx for pod openshift-network-diagnostics/network-check-target-nq29s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 11:48:22.507466 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:22.507396 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1598d282-cf11-48e5-b3e5-d3feac72576b-kube-api-access-9gttx podName:1598d282-cf11-48e5-b3e5-d3feac72576b nodeName:}" failed. No retries permitted until 2026-09-11 11:48:54.50737111 +0000 UTC m=+66.272344277 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-9gttx" (UniqueName: "kubernetes.io/projected/1598d282-cf11-48e5-b3e5-d3feac72576b-kube-api-access-9gttx") pod "network-check-target-nq29s" (UID: "1598d282-cf11-48e5-b3e5-d3feac72576b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 11:48:22.753206 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.752894 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nq29s" Sep 11 11:48:22.753206 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.752897 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x982g" Sep 11 11:48:22.753206 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.752904 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfx78" Sep 11 11:48:22.757255 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.756460 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xblbl\"" Sep 11 11:48:22.757255 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.756517 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 11 11:48:22.757438 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.757241 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 11 11:48:22.757438 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.757246 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 11 11:48:22.757438 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.757324 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 11 11:48:22.758114 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.758091 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5sht6\"" Sep 11 11:48:22.827427 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.827400 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5qrzc"] Sep 11 11:48:22.831514 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:48:22.831485 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1190be0_add2_48ea_ad87_5d7271697bfa.slice/crio-3481c4b3a47a31ea51fa82e2d7f8b28b875857523b09f3a83ffdced203955bac WatchSource:0}: Error finding container 3481c4b3a47a31ea51fa82e2d7f8b28b875857523b09f3a83ffdced203955bac: Status 404 returned error can't find the container with id 3481c4b3a47a31ea51fa82e2d7f8b28b875857523b09f3a83ffdced203955bac Sep 11 11:48:22.910115 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.910081 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-registry-tls\") pod \"image-registry-9f4c766d4-g58wm\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" Sep 11 11:48:22.910288 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:22.910259 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 11 11:48:22.910288 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.910264 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6673d6af-e1b2-443b-83c2-7dc7b653b9d0-metrics-tls\") pod \"dns-default-9f582\" (UID: \"6673d6af-e1b2-443b-83c2-7dc7b653b9d0\") " pod="openshift-dns/dns-default-9f582" Sep 11 11:48:22.910288 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:22.910281 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9f4c766d4-g58wm: secret "image-registry-tls" not found Sep 11 11:48:22.910443 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:22.910343 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-registry-tls podName:60eec8ae-8c59-40f6-820a-9671b9ec919c nodeName:}" failed. No retries permitted until 2026-09-11 11:48:23.910323975 +0000 UTC m=+35.675297132 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-registry-tls") pod "image-registry-9f4c766d4-g58wm" (UID: "60eec8ae-8c59-40f6-820a-9671b9ec919c") : secret "image-registry-tls" not found Sep 11 11:48:22.910443 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:22.910421 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 11 11:48:22.910547 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:22.910464 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6673d6af-e1b2-443b-83c2-7dc7b653b9d0-metrics-tls podName:6673d6af-e1b2-443b-83c2-7dc7b653b9d0 nodeName:}" failed. No retries permitted until 2026-09-11 11:48:23.910450299 +0000 UTC m=+35.675423470 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6673d6af-e1b2-443b-83c2-7dc7b653b9d0-metrics-tls") pod "dns-default-9f582" (UID: "6673d6af-e1b2-443b-83c2-7dc7b653b9d0") : secret "dns-default-metrics-tls" not found Sep 11 11:48:22.910948 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:22.910929 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5qrzc" event={"ID":"d1190be0-add2-48ea-ad87-5d7271697bfa","Type":"ContainerStarted","Data":"3481c4b3a47a31ea51fa82e2d7f8b28b875857523b09f3a83ffdced203955bac"} Sep 11 11:48:23.919474 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:23.919445 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-registry-tls\") pod \"image-registry-9f4c766d4-g58wm\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" Sep 11 11:48:23.919895 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:23.919535 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6673d6af-e1b2-443b-83c2-7dc7b653b9d0-metrics-tls\") pod \"dns-default-9f582\" (UID: \"6673d6af-e1b2-443b-83c2-7dc7b653b9d0\") " pod="openshift-dns/dns-default-9f582" Sep 11 11:48:23.919895 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:23.919611 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 11 11:48:23.919895 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:23.919632 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9f4c766d4-g58wm: secret "image-registry-tls" not found Sep 11 11:48:23.919895 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:23.919643 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 11 11:48:23.919895 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:23.919685 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6673d6af-e1b2-443b-83c2-7dc7b653b9d0-metrics-tls podName:6673d6af-e1b2-443b-83c2-7dc7b653b9d0 nodeName:}" failed. No retries permitted until 2026-09-11 11:48:25.91967263 +0000 UTC m=+37.684645784 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6673d6af-e1b2-443b-83c2-7dc7b653b9d0-metrics-tls") pod "dns-default-9f582" (UID: "6673d6af-e1b2-443b-83c2-7dc7b653b9d0") : secret "dns-default-metrics-tls" not found Sep 11 11:48:23.919895 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:23.919703 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-registry-tls podName:60eec8ae-8c59-40f6-820a-9671b9ec919c nodeName:}" failed. No retries permitted until 2026-09-11 11:48:25.919693825 +0000 UTC m=+37.684666978 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-registry-tls") pod "image-registry-9f4c766d4-g58wm" (UID: "60eec8ae-8c59-40f6-820a-9671b9ec919c") : secret "image-registry-tls" not found Sep 11 11:48:24.920729 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:24.920685 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cs8ww" event={"ID":"091f9985-8446-4191-9ec0-9f40b1d1a5fb","Type":"ContainerStarted","Data":"33128ab1d52a59c7c8bd8f443a7cbc81c949071914e3aea31bd9267f418058c5"} Sep 11 11:48:25.926524 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:25.926493 2566 generic.go:356] "Generic (PLEG): container finished" podID="091f9985-8446-4191-9ec0-9f40b1d1a5fb" containerID="33128ab1d52a59c7c8bd8f443a7cbc81c949071914e3aea31bd9267f418058c5" exitCode=0 Sep 11 11:48:25.926927 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:25.926559 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cs8ww" event={"ID":"091f9985-8446-4191-9ec0-9f40b1d1a5fb","Type":"ContainerDied","Data":"33128ab1d52a59c7c8bd8f443a7cbc81c949071914e3aea31bd9267f418058c5"} Sep 11 11:48:25.934894 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:25.934860 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6673d6af-e1b2-443b-83c2-7dc7b653b9d0-metrics-tls\") pod \"dns-default-9f582\" (UID: \"6673d6af-e1b2-443b-83c2-7dc7b653b9d0\") " pod="openshift-dns/dns-default-9f582" Sep 11 11:48:25.934951 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:25.934923 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-registry-tls\") pod \"image-registry-9f4c766d4-g58wm\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" Sep 11 11:48:25.934995 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:25.934968 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 11 11:48:25.935033 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:25.935014 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6673d6af-e1b2-443b-83c2-7dc7b653b9d0-metrics-tls podName:6673d6af-e1b2-443b-83c2-7dc7b653b9d0 nodeName:}" failed. No retries permitted until 2026-09-11 11:48:29.935001655 +0000 UTC m=+41.699974807 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6673d6af-e1b2-443b-83c2-7dc7b653b9d0-metrics-tls") pod "dns-default-9f582" (UID: "6673d6af-e1b2-443b-83c2-7dc7b653b9d0") : secret "dns-default-metrics-tls" not found Sep 11 11:48:25.935079 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:25.935046 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 11 11:48:25.935079 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:25.935058 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9f4c766d4-g58wm: secret "image-registry-tls" not found Sep 11 11:48:25.935146 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:25.935097 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-registry-tls podName:60eec8ae-8c59-40f6-820a-9671b9ec919c nodeName:}" failed. No retries permitted until 2026-09-11 11:48:29.935086246 +0000 UTC m=+41.700059398 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-registry-tls") pod "image-registry-9f4c766d4-g58wm" (UID: "60eec8ae-8c59-40f6-820a-9671b9ec919c") : secret "image-registry-tls" not found Sep 11 11:48:26.932234 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:26.932154 2566 generic.go:356] "Generic (PLEG): container finished" podID="091f9985-8446-4191-9ec0-9f40b1d1a5fb" containerID="6bc5f8fe7f2058a444a4bf37f7f118e34bfbc59f1898a5c3070e17f1cb2f6448" exitCode=0 Sep 11 11:48:26.932234 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:26.932218 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cs8ww" event={"ID":"091f9985-8446-4191-9ec0-9f40b1d1a5fb","Type":"ContainerDied","Data":"6bc5f8fe7f2058a444a4bf37f7f118e34bfbc59f1898a5c3070e17f1cb2f6448"} Sep 11 11:48:27.763226 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:27.763197 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n9pc8_f90862ea-2909-4701-ab56-fe2d3fbb83f8/konnectivity-agent/0.log" limit=1024 Sep 11 11:48:27.867052 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:27.867018 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n9pc8_f90862ea-2909-4701-ab56-fe2d3fbb83f8/konnectivity-agent/0.log" limit=1024 Sep 11 11:48:27.937962 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:27.937921 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cs8ww" event={"ID":"091f9985-8446-4191-9ec0-9f40b1d1a5fb","Type":"ContainerStarted","Data":"a0d84220c838f9b1b4ada93a64f553647cced51389a276bd20a949a601c5d4a2"} Sep 11 11:48:27.939304 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:27.939285 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5qrzc" event={"ID":"d1190be0-add2-48ea-ad87-5d7271697bfa","Type":"ContainerStarted","Data":"65ca577004e1ec68c65c68332b4e0c7f58e8750a1ec5b391c1cf52fffdaa22e5"} Sep 11 11:48:27.967857 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:27.967806 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-cs8ww" podStartSLOduration=8.856464095 podStartE2EDuration="38.967792703s" totalImagesPullingTime="30.111328608s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:47:49 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:48:27.966166531 +0000 UTC m=+39.731139708" watchObservedRunningTime="2026-09-11 11:48:27.967792703 +0000 UTC m=+39.732765878" Sep 11 11:48:27.986088 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:27.986046 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5qrzc" podStartSLOduration=1.677459305 podStartE2EDuration="5.986033465s" totalImagesPullingTime="4.30857416s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:48:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:48:27.985155037 +0000 UTC m=+39.750128212" watchObservedRunningTime="2026-09-11 11:48:27.986033465 +0000 UTC m=+39.751006637" Sep 11 11:48:28.100180 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.100147 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n9pc8_f90862ea-2909-4701-ab56-fe2d3fbb83f8/konnectivity-agent/0.log" limit=1024 Sep 11 11:48:28.160616 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.160591 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84cbf47796-f5lw9"] Sep 11 11:48:28.182142 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.182117 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84cbf47796-f5lw9"] Sep 11 11:48:28.182300 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.182198 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84cbf47796-f5lw9" Sep 11 11:48:28.184509 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.184488 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-v7tkv\"" Sep 11 11:48:28.184620 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.184488 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 11 11:48:28.184748 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.184731 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 11 11:48:28.185061 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.185044 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Sep 11 11:48:28.185177 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.185161 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 11 11:48:28.190508 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.190485 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d44464d58-zfgcj"] Sep 11 11:48:28.190599 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.190554 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n9pc8_f90862ea-2909-4701-ab56-fe2d3fbb83f8/konnectivity-agent/0.log" limit=1024 Sep 11 11:48:28.216267 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.216249 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d44464d58-zfgcj"] Sep 11 11:48:28.216359 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.216332 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d44464d58-zfgcj" Sep 11 11:48:28.219181 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.219162 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Sep 11 11:48:28.240101 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.240079 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6467cf759d-jvf6s"] Sep 11 11:48:28.264458 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.264441 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6467cf759d-jvf6s"] Sep 11 11:48:28.264559 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.264519 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6467cf759d-jvf6s" Sep 11 11:48:28.267587 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.267569 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Sep 11 11:48:28.267944 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.267929 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Sep 11 11:48:28.268426 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.268405 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Sep 11 11:48:28.268590 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.268567 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Sep 11 11:48:28.359281 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.359196 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/60572541-4256-4131-9831-50ef928135e1-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-84cbf47796-f5lw9\" (UID: \"60572541-4256-4131-9831-50ef928135e1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84cbf47796-f5lw9" Sep 11 11:48:28.359281 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.359234 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/60ce193d-c29c-41eb-a8f2-e0148c0584ef-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6467cf759d-jvf6s\" (UID: \"60ce193d-c29c-41eb-a8f2-e0148c0584ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6467cf759d-jvf6s" Sep 11 11:48:28.359281 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.359253 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/60ce193d-c29c-41eb-a8f2-e0148c0584ef-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6467cf759d-jvf6s\" (UID: \"60ce193d-c29c-41eb-a8f2-e0148c0584ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6467cf759d-jvf6s" Sep 11 11:48:28.359281 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.359271 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/60ce193d-c29c-41eb-a8f2-e0148c0584ef-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6467cf759d-jvf6s\" (UID: \"60ce193d-c29c-41eb-a8f2-e0148c0584ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6467cf759d-jvf6s" Sep 11 11:48:28.359513 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.359385 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/60ce193d-c29c-41eb-a8f2-e0148c0584ef-hub\") pod \"cluster-proxy-proxy-agent-6467cf759d-jvf6s\" (UID: \"60ce193d-c29c-41eb-a8f2-e0148c0584ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6467cf759d-jvf6s" Sep 11 11:48:28.359513 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.359420 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/60ce193d-c29c-41eb-a8f2-e0148c0584ef-ca\") pod \"cluster-proxy-proxy-agent-6467cf759d-jvf6s\" (UID: \"60ce193d-c29c-41eb-a8f2-e0148c0584ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6467cf759d-jvf6s" Sep 11 11:48:28.359513 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.359438 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d1d3ab73-3bf4-494e-bd65-b92790da8f33-tmp\") pod \"klusterlet-addon-workmgr-6d44464d58-zfgcj\" (UID: \"d1d3ab73-3bf4-494e-bd65-b92790da8f33\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d44464d58-zfgcj" Sep 11 11:48:28.359513 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.359453 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d1d3ab73-3bf4-494e-bd65-b92790da8f33-klusterlet-config\") pod \"klusterlet-addon-workmgr-6d44464d58-zfgcj\" (UID: \"d1d3ab73-3bf4-494e-bd65-b92790da8f33\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d44464d58-zfgcj" Sep 11 11:48:28.359627 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.359521 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsjhn\" (UniqueName: \"kubernetes.io/projected/60572541-4256-4131-9831-50ef928135e1-kube-api-access-hsjhn\") pod \"managed-serviceaccount-addon-agent-84cbf47796-f5lw9\" (UID: \"60572541-4256-4131-9831-50ef928135e1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84cbf47796-f5lw9" Sep 11 11:48:28.359627 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.359547 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbv9x\" (UniqueName: \"kubernetes.io/projected/60ce193d-c29c-41eb-a8f2-e0148c0584ef-kube-api-access-lbv9x\") pod \"cluster-proxy-proxy-agent-6467cf759d-jvf6s\" (UID: \"60ce193d-c29c-41eb-a8f2-e0148c0584ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6467cf759d-jvf6s" Sep 11 11:48:28.359627 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.359564 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhxgc\" (UniqueName: \"kubernetes.io/projected/d1d3ab73-3bf4-494e-bd65-b92790da8f33-kube-api-access-lhxgc\") pod \"klusterlet-addon-workmgr-6d44464d58-zfgcj\" (UID: \"d1d3ab73-3bf4-494e-bd65-b92790da8f33\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d44464d58-zfgcj" Sep 11 11:48:28.460545 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.460509 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/60572541-4256-4131-9831-50ef928135e1-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-84cbf47796-f5lw9\" (UID: \"60572541-4256-4131-9831-50ef928135e1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84cbf47796-f5lw9" Sep 11 11:48:28.460545 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.460547 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/60ce193d-c29c-41eb-a8f2-e0148c0584ef-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6467cf759d-jvf6s\" (UID: \"60ce193d-c29c-41eb-a8f2-e0148c0584ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6467cf759d-jvf6s" Sep 11 11:48:28.460769 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.460567 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/60ce193d-c29c-41eb-a8f2-e0148c0584ef-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6467cf759d-jvf6s\" (UID: \"60ce193d-c29c-41eb-a8f2-e0148c0584ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6467cf759d-jvf6s" Sep 11 11:48:28.460769 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.460588 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/60ce193d-c29c-41eb-a8f2-e0148c0584ef-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6467cf759d-jvf6s\" (UID: \"60ce193d-c29c-41eb-a8f2-e0148c0584ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6467cf759d-jvf6s" Sep 11 11:48:28.460769 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.460647 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/60ce193d-c29c-41eb-a8f2-e0148c0584ef-hub\") pod \"cluster-proxy-proxy-agent-6467cf759d-jvf6s\" (UID: \"60ce193d-c29c-41eb-a8f2-e0148c0584ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6467cf759d-jvf6s" Sep 11 11:48:28.460769 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.460676 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/60ce193d-c29c-41eb-a8f2-e0148c0584ef-ca\") pod \"cluster-proxy-proxy-agent-6467cf759d-jvf6s\" (UID: \"60ce193d-c29c-41eb-a8f2-e0148c0584ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6467cf759d-jvf6s" Sep 11 11:48:28.460769 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.460698 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d1d3ab73-3bf4-494e-bd65-b92790da8f33-tmp\") pod \"klusterlet-addon-workmgr-6d44464d58-zfgcj\" (UID: \"d1d3ab73-3bf4-494e-bd65-b92790da8f33\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d44464d58-zfgcj" Sep 11 11:48:28.460769 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.460719 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d1d3ab73-3bf4-494e-bd65-b92790da8f33-klusterlet-config\") pod \"klusterlet-addon-workmgr-6d44464d58-zfgcj\" (UID: \"d1d3ab73-3bf4-494e-bd65-b92790da8f33\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d44464d58-zfgcj" Sep 11 11:48:28.460769 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.460758 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsjhn\" (UniqueName: \"kubernetes.io/projected/60572541-4256-4131-9831-50ef928135e1-kube-api-access-hsjhn\") pod \"managed-serviceaccount-addon-agent-84cbf47796-f5lw9\" (UID: \"60572541-4256-4131-9831-50ef928135e1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84cbf47796-f5lw9" Sep 11 11:48:28.461090 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.460795 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbv9x\" (UniqueName: \"kubernetes.io/projected/60ce193d-c29c-41eb-a8f2-e0148c0584ef-kube-api-access-lbv9x\") pod \"cluster-proxy-proxy-agent-6467cf759d-jvf6s\" (UID: \"60ce193d-c29c-41eb-a8f2-e0148c0584ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6467cf759d-jvf6s" Sep 11 11:48:28.461090 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.460836 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhxgc\" (UniqueName: \"kubernetes.io/projected/d1d3ab73-3bf4-494e-bd65-b92790da8f33-kube-api-access-lhxgc\") pod \"klusterlet-addon-workmgr-6d44464d58-zfgcj\" (UID: \"d1d3ab73-3bf4-494e-bd65-b92790da8f33\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d44464d58-zfgcj" Sep 11 11:48:28.461090 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.461071 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d1d3ab73-3bf4-494e-bd65-b92790da8f33-tmp\") pod \"klusterlet-addon-workmgr-6d44464d58-zfgcj\" (UID: \"d1d3ab73-3bf4-494e-bd65-b92790da8f33\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d44464d58-zfgcj" Sep 11 11:48:28.461415 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.461389 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/60ce193d-c29c-41eb-a8f2-e0148c0584ef-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6467cf759d-jvf6s\" (UID: \"60ce193d-c29c-41eb-a8f2-e0148c0584ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6467cf759d-jvf6s" Sep 11 11:48:28.465156 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.465131 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/60ce193d-c29c-41eb-a8f2-e0148c0584ef-ca\") pod \"cluster-proxy-proxy-agent-6467cf759d-jvf6s\" (UID: \"60ce193d-c29c-41eb-a8f2-e0148c0584ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6467cf759d-jvf6s" Sep 11 11:48:28.465248 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.465169 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/60ce193d-c29c-41eb-a8f2-e0148c0584ef-hub\") pod \"cluster-proxy-proxy-agent-6467cf759d-jvf6s\" (UID: \"60ce193d-c29c-41eb-a8f2-e0148c0584ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6467cf759d-jvf6s" Sep 11 11:48:28.465248 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.465209 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/60ce193d-c29c-41eb-a8f2-e0148c0584ef-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6467cf759d-jvf6s\" (UID: \"60ce193d-c29c-41eb-a8f2-e0148c0584ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6467cf759d-jvf6s" Sep 11 11:48:28.465343 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.465319 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/60ce193d-c29c-41eb-a8f2-e0148c0584ef-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6467cf759d-jvf6s\" (UID: \"60ce193d-c29c-41eb-a8f2-e0148c0584ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6467cf759d-jvf6s" Sep 11 11:48:28.465376 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.465352 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d1d3ab73-3bf4-494e-bd65-b92790da8f33-klusterlet-config\") pod \"klusterlet-addon-workmgr-6d44464d58-zfgcj\" (UID: \"d1d3ab73-3bf4-494e-bd65-b92790da8f33\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d44464d58-zfgcj" Sep 11 11:48:28.465457 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.465443 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/60572541-4256-4131-9831-50ef928135e1-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-84cbf47796-f5lw9\" (UID: \"60572541-4256-4131-9831-50ef928135e1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84cbf47796-f5lw9" Sep 11 11:48:28.470204 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.470180 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhxgc\" (UniqueName: \"kubernetes.io/projected/d1d3ab73-3bf4-494e-bd65-b92790da8f33-kube-api-access-lhxgc\") pod \"klusterlet-addon-workmgr-6d44464d58-zfgcj\" (UID: \"d1d3ab73-3bf4-494e-bd65-b92790da8f33\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d44464d58-zfgcj" Sep 11 11:48:28.470445 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.470429 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsjhn\" (UniqueName: \"kubernetes.io/projected/60572541-4256-4131-9831-50ef928135e1-kube-api-access-hsjhn\") pod \"managed-serviceaccount-addon-agent-84cbf47796-f5lw9\" (UID: \"60572541-4256-4131-9831-50ef928135e1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84cbf47796-f5lw9" Sep 11 11:48:28.470483 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.470458 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbv9x\" (UniqueName: \"kubernetes.io/projected/60ce193d-c29c-41eb-a8f2-e0148c0584ef-kube-api-access-lbv9x\") pod \"cluster-proxy-proxy-agent-6467cf759d-jvf6s\" (UID: \"60ce193d-c29c-41eb-a8f2-e0148c0584ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6467cf759d-jvf6s" Sep 11 11:48:28.502833 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.502808 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84cbf47796-f5lw9" Sep 11 11:48:28.523969 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.523951 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d44464d58-zfgcj" Sep 11 11:48:28.575590 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.575561 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6467cf759d-jvf6s" Sep 11 11:48:28.809772 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.809738 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84cbf47796-f5lw9"] Sep 11 11:48:28.814716 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:48:28.814685 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod60572541_4256_4131_9831_50ef928135e1.slice/crio-025031e23c12e14c674c5e7eaf53bda1ae11436ecb8b3b41cffb2dddf001313d WatchSource:0}: Error finding container 025031e23c12e14c674c5e7eaf53bda1ae11436ecb8b3b41cffb2dddf001313d: Status 404 returned error can't find the container with id 025031e23c12e14c674c5e7eaf53bda1ae11436ecb8b3b41cffb2dddf001313d Sep 11 11:48:28.839188 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.839162 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d44464d58-zfgcj"] Sep 11 11:48:28.841978 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:48:28.841953 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1d3ab73_3bf4_494e_bd65_b92790da8f33.slice/crio-547ddd493f9e8c0b0e46409b6a920fbc4e038a74d12547dd197eae95f633ab1e WatchSource:0}: Error finding container 547ddd493f9e8c0b0e46409b6a920fbc4e038a74d12547dd197eae95f633ab1e: Status 404 returned error can't find the container with id 547ddd493f9e8c0b0e46409b6a920fbc4e038a74d12547dd197eae95f633ab1e Sep 11 11:48:28.875958 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.875881 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6467cf759d-jvf6s"] Sep 11 11:48:28.879443 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:48:28.879419 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod60ce193d_c29c_41eb_a8f2_e0148c0584ef.slice/crio-90b38438e437992a13c0a1740ebe85b90eb502df87a0354147cda2824f4ca635 WatchSource:0}: Error finding container 90b38438e437992a13c0a1740ebe85b90eb502df87a0354147cda2824f4ca635: Status 404 returned error can't find the container with id 90b38438e437992a13c0a1740ebe85b90eb502df87a0354147cda2824f4ca635 Sep 11 11:48:28.942490 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.942463 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6467cf759d-jvf6s" event={"ID":"60ce193d-c29c-41eb-a8f2-e0148c0584ef","Type":"ContainerStarted","Data":"90b38438e437992a13c0a1740ebe85b90eb502df87a0354147cda2824f4ca635"} Sep 11 11:48:28.943586 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.943563 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84cbf47796-f5lw9" event={"ID":"60572541-4256-4131-9831-50ef928135e1","Type":"ContainerStarted","Data":"025031e23c12e14c674c5e7eaf53bda1ae11436ecb8b3b41cffb2dddf001313d"} Sep 11 11:48:28.944706 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:28.944681 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d44464d58-zfgcj" event={"ID":"d1d3ab73-3bf4-494e-bd65-b92790da8f33","Type":"ContainerStarted","Data":"547ddd493f9e8c0b0e46409b6a920fbc4e038a74d12547dd197eae95f633ab1e"} Sep 11 11:48:29.973711 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:29.973676 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6673d6af-e1b2-443b-83c2-7dc7b653b9d0-metrics-tls\") pod \"dns-default-9f582\" (UID: \"6673d6af-e1b2-443b-83c2-7dc7b653b9d0\") " pod="openshift-dns/dns-default-9f582" Sep 11 11:48:29.974174 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:29.973742 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-registry-tls\") pod \"image-registry-9f4c766d4-g58wm\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" Sep 11 11:48:29.974174 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:29.973890 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 11 11:48:29.974174 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:29.973905 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9f4c766d4-g58wm: secret "image-registry-tls" not found Sep 11 11:48:29.974174 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:29.973985 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-registry-tls podName:60eec8ae-8c59-40f6-820a-9671b9ec919c nodeName:}" failed. No retries permitted until 2026-09-11 11:48:37.973942347 +0000 UTC m=+49.738915517 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-registry-tls") pod "image-registry-9f4c766d4-g58wm" (UID: "60eec8ae-8c59-40f6-820a-9671b9ec919c") : secret "image-registry-tls" not found Sep 11 11:48:29.974586 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:29.974454 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 11 11:48:29.974586 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:29.974511 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6673d6af-e1b2-443b-83c2-7dc7b653b9d0-metrics-tls podName:6673d6af-e1b2-443b-83c2-7dc7b653b9d0 nodeName:}" failed. No retries permitted until 2026-09-11 11:48:37.974495035 +0000 UTC m=+49.739468193 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6673d6af-e1b2-443b-83c2-7dc7b653b9d0-metrics-tls") pod "dns-default-9f582" (UID: "6673d6af-e1b2-443b-83c2-7dc7b653b9d0") : secret "dns-default-metrics-tls" not found Sep 11 11:48:32.662905 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:32.662848 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n9pc8_f90862ea-2909-4701-ab56-fe2d3fbb83f8/konnectivity-agent/0.log" limit=1024 Sep 11 11:48:32.763404 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:32.763369 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n9pc8_f90862ea-2909-4701-ab56-fe2d3fbb83f8/konnectivity-agent/0.log" limit=1024 Sep 11 11:48:36.412271 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:36.412245 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n9pc8_f90862ea-2909-4701-ab56-fe2d3fbb83f8/konnectivity-agent/0.log" limit=1024 Sep 11 11:48:36.512842 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:36.512813 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n9pc8_f90862ea-2909-4701-ab56-fe2d3fbb83f8/konnectivity-agent/0.log" limit=1024 Sep 11 11:48:36.971409 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:36.971335 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d44464d58-zfgcj" event={"ID":"d1d3ab73-3bf4-494e-bd65-b92790da8f33","Type":"ContainerStarted","Data":"a23de5b596bcd30cc69af66df79e38a88df42619111531b07904f3047895b296"} Sep 11 11:48:36.971556 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:36.971520 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d44464d58-zfgcj" Sep 11 11:48:36.973555 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:36.973522 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d44464d58-zfgcj" Sep 11 11:48:36.973721 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:36.973701 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6467cf759d-jvf6s" event={"ID":"60ce193d-c29c-41eb-a8f2-e0148c0584ef","Type":"ContainerStarted","Data":"9bb80a01725f4860b98ef4a90e8c1a06139605bf9c5f256816a16c0ffbd1d9e5"} Sep 11 11:48:36.973810 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:36.973730 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6467cf759d-jvf6s" event={"ID":"60ce193d-c29c-41eb-a8f2-e0148c0584ef","Type":"ContainerStarted","Data":"212a23deb96b8af3404988ce80c26e1303430d5470c33d2f2afba8ea4cb633f5"} Sep 11 11:48:36.973810 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:36.973746 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6467cf759d-jvf6s" event={"ID":"60ce193d-c29c-41eb-a8f2-e0148c0584ef","Type":"ContainerStarted","Data":"049f1407660839e541642acd85912701b7348b107697f26449b4c1f4b04d0174"} Sep 11 11:48:36.975145 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:36.975122 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84cbf47796-f5lw9" event={"ID":"60572541-4256-4131-9831-50ef928135e1","Type":"ContainerStarted","Data":"b1c46143990ccd73b7a0874a8d7d2e0b24b0817c8a7ab4da2c0a43e29d73ff92"} Sep 11 11:48:37.034725 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:37.034673 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d44464d58-zfgcj" podStartSLOduration=1.5706871119999999 podStartE2EDuration="9.03465645s" totalImagesPullingTime="7.463969338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:48:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:48:37.000197026 +0000 UTC m=+48.765170219" watchObservedRunningTime="2026-09-11 11:48:37.03465645 +0000 UTC m=+48.799629625" Sep 11 11:48:37.066402 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:37.066359 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84cbf47796-f5lw9" podStartSLOduration=1.576034567 podStartE2EDuration="9.066343213s" totalImagesPullingTime="7.490308646s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:48:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:48:37.065102158 +0000 UTC m=+48.830075332" watchObservedRunningTime="2026-09-11 11:48:37.066343213 +0000 UTC m=+48.831316387" Sep 11 11:48:37.066549 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:37.066425 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6467cf759d-jvf6s" podStartSLOduration=1.624705741 podStartE2EDuration="9.066420545s" totalImagesPullingTime="7.441714804s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:48:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:48:37.035984677 +0000 UTC m=+48.800957853" watchObservedRunningTime="2026-09-11 11:48:37.066420545 +0000 UTC m=+48.831393719" Sep 11 11:48:37.135518 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:37.135474 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n9pc8_f90862ea-2909-4701-ab56-fe2d3fbb83f8/konnectivity-agent/0.log" limit=1024 Sep 11 11:48:37.564671 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:37.564627 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n9pc8_f90862ea-2909-4701-ab56-fe2d3fbb83f8/konnectivity-agent/0.log" limit=1024 Sep 11 11:48:38.047343 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:38.047307 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6673d6af-e1b2-443b-83c2-7dc7b653b9d0-metrics-tls\") pod \"dns-default-9f582\" (UID: \"6673d6af-e1b2-443b-83c2-7dc7b653b9d0\") " pod="openshift-dns/dns-default-9f582" Sep 11 11:48:38.047597 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:38.047576 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 11 11:48:38.047711 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:38.047609 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-registry-tls\") pod \"image-registry-9f4c766d4-g58wm\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" Sep 11 11:48:38.047711 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:38.047645 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6673d6af-e1b2-443b-83c2-7dc7b653b9d0-metrics-tls podName:6673d6af-e1b2-443b-83c2-7dc7b653b9d0 nodeName:}" failed. No retries permitted until 2026-09-11 11:48:54.047623196 +0000 UTC m=+65.812596363 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6673d6af-e1b2-443b-83c2-7dc7b653b9d0-metrics-tls") pod "dns-default-9f582" (UID: "6673d6af-e1b2-443b-83c2-7dc7b653b9d0") : secret "dns-default-metrics-tls" not found Sep 11 11:48:38.047711 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:38.047692 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 11 11:48:38.047711 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:38.047705 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9f4c766d4-g58wm: secret "image-registry-tls" not found Sep 11 11:48:38.048108 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:38.047767 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-registry-tls podName:60eec8ae-8c59-40f6-820a-9671b9ec919c nodeName:}" failed. No retries permitted until 2026-09-11 11:48:54.047749824 +0000 UTC m=+65.812722992 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-registry-tls") pod "image-registry-9f4c766d4-g58wm" (UID: "60eec8ae-8c59-40f6-820a-9671b9ec919c") : secret "image-registry-tls" not found Sep 11 11:48:48.910964 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:48.910934 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-25t6q" Sep 11 11:48:54.066946 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:54.066902 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6673d6af-e1b2-443b-83c2-7dc7b653b9d0-metrics-tls\") pod \"dns-default-9f582\" (UID: \"6673d6af-e1b2-443b-83c2-7dc7b653b9d0\") " pod="openshift-dns/dns-default-9f582" Sep 11 11:48:54.067327 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:54.066963 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-registry-tls\") pod \"image-registry-9f4c766d4-g58wm\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" Sep 11 11:48:54.067327 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:54.067056 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 11 11:48:54.067327 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:54.067068 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9f4c766d4-g58wm: secret "image-registry-tls" not found Sep 11 11:48:54.067327 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:54.067056 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 11 11:48:54.067327 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:54.067111 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-registry-tls podName:60eec8ae-8c59-40f6-820a-9671b9ec919c nodeName:}" failed. No retries permitted until 2026-09-11 11:49:26.067098346 +0000 UTC m=+97.832071499 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-registry-tls") pod "image-registry-9f4c766d4-g58wm" (UID: "60eec8ae-8c59-40f6-820a-9671b9ec919c") : secret "image-registry-tls" not found Sep 11 11:48:54.067327 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:54.067213 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6673d6af-e1b2-443b-83c2-7dc7b653b9d0-metrics-tls podName:6673d6af-e1b2-443b-83c2-7dc7b653b9d0 nodeName:}" failed. No retries permitted until 2026-09-11 11:49:26.067194544 +0000 UTC m=+97.832167705 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6673d6af-e1b2-443b-83c2-7dc7b653b9d0-metrics-tls") pod "dns-default-9f582" (UID: "6673d6af-e1b2-443b-83c2-7dc7b653b9d0") : secret "dns-default-metrics-tls" not found Sep 11 11:48:54.469745 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:54.469650 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4fd9da2-13c7-4b85-9c89-f10a28d43049-metrics-certs\") pod \"network-metrics-daemon-x982g\" (UID: \"d4fd9da2-13c7-4b85-9c89-f10a28d43049\") " pod="openshift-multus/network-metrics-daemon-x982g" Sep 11 11:48:54.469745 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:54.469723 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2dc78d6d-cf10-4087-a38b-0ded1057f222-original-pull-secret\") pod \"global-pull-secret-syncer-tfx78\" (UID: \"2dc78d6d-cf10-4087-a38b-0ded1057f222\") " pod="kube-system/global-pull-secret-syncer-tfx78" Sep 11 11:48:54.472485 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:54.472465 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 11 11:48:54.472761 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:54.472746 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 11 11:48:54.479896 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:54.479878 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 11 11:48:54.479956 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:48:54.479944 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4fd9da2-13c7-4b85-9c89-f10a28d43049-metrics-certs podName:d4fd9da2-13c7-4b85-9c89-f10a28d43049 nodeName:}" failed. No retries permitted until 2026-09-11 11:49:58.479923073 +0000 UTC m=+130.244896241 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4fd9da2-13c7-4b85-9c89-f10a28d43049-metrics-certs") pod "network-metrics-daemon-x982g" (UID: "d4fd9da2-13c7-4b85-9c89-f10a28d43049") : secret "metrics-daemon-secret" not found Sep 11 11:48:54.482042 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:54.482025 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2dc78d6d-cf10-4087-a38b-0ded1057f222-original-pull-secret\") pod \"global-pull-secret-syncer-tfx78\" (UID: \"2dc78d6d-cf10-4087-a38b-0ded1057f222\") " pod="kube-system/global-pull-secret-syncer-tfx78" Sep 11 11:48:54.570505 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:54.570459 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gttx\" (UniqueName: \"kubernetes.io/projected/1598d282-cf11-48e5-b3e5-d3feac72576b-kube-api-access-9gttx\") pod \"network-check-target-nq29s\" (UID: \"1598d282-cf11-48e5-b3e5-d3feac72576b\") " pod="openshift-network-diagnostics/network-check-target-nq29s" Sep 11 11:48:54.573506 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:54.573488 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 11 11:48:54.575597 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:54.575581 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfx78" Sep 11 11:48:54.584566 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:54.584359 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 11 11:48:54.593657 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:54.593620 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gttx\" (UniqueName: \"kubernetes.io/projected/1598d282-cf11-48e5-b3e5-d3feac72576b-kube-api-access-9gttx\") pod \"network-check-target-nq29s\" (UID: \"1598d282-cf11-48e5-b3e5-d3feac72576b\") " pod="openshift-network-diagnostics/network-check-target-nq29s" Sep 11 11:48:54.845521 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:54.845492 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tfx78"] Sep 11 11:48:54.848594 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:48:54.848572 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dc78d6d_cf10_4087_a38b_0ded1057f222.slice/crio-3753749e98fd45f884d90a1a332508dc0f0944a0d24cc1e277cf070cba69d204 WatchSource:0}: Error finding container 3753749e98fd45f884d90a1a332508dc0f0944a0d24cc1e277cf070cba69d204: Status 404 returned error can't find the container with id 3753749e98fd45f884d90a1a332508dc0f0944a0d24cc1e277cf070cba69d204 Sep 11 11:48:54.871559 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:54.871538 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xblbl\"" Sep 11 11:48:54.878827 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:54.878809 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nq29s" Sep 11 11:48:55.027414 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:55.027374 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tfx78" event={"ID":"2dc78d6d-cf10-4087-a38b-0ded1057f222","Type":"ContainerStarted","Data":"3753749e98fd45f884d90a1a332508dc0f0944a0d24cc1e277cf070cba69d204"} Sep 11 11:48:55.149948 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:55.149918 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nq29s"] Sep 11 11:48:55.152169 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:48:55.152142 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1598d282_cf11_48e5_b3e5_d3feac72576b.slice/crio-76b5fd27aa198bd0301a22abfed61c16b48f58273853fc66bcc3415d9556727f WatchSource:0}: Error finding container 76b5fd27aa198bd0301a22abfed61c16b48f58273853fc66bcc3415d9556727f: Status 404 returned error can't find the container with id 76b5fd27aa198bd0301a22abfed61c16b48f58273853fc66bcc3415d9556727f Sep 11 11:48:56.032347 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:48:56.032310 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nq29s" event={"ID":"1598d282-cf11-48e5-b3e5-d3feac72576b","Type":"ContainerStarted","Data":"76b5fd27aa198bd0301a22abfed61c16b48f58273853fc66bcc3415d9556727f"} Sep 11 11:49:00.046661 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:49:00.046623 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tfx78" event={"ID":"2dc78d6d-cf10-4087-a38b-0ded1057f222","Type":"ContainerStarted","Data":"6242a3e5480034761b4e48fea8ba81c5962ddaeacca1caab84a28c00f1cb2981"} Sep 11 11:49:00.048257 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:49:00.048234 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nq29s" event={"ID":"1598d282-cf11-48e5-b3e5-d3feac72576b","Type":"ContainerStarted","Data":"17739ee33b433ce1741ea84aa439f87a1c036093073474035251f66066c72325"} Sep 11 11:49:00.048394 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:49:00.048376 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-nq29s" Sep 11 11:49:00.067655 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:49:00.067609 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-tfx78" podStartSLOduration=65.974559924 podStartE2EDuration="1m11.067593535s" totalImagesPullingTime="5.093033611s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:47:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:49:00.067090455 +0000 UTC m=+71.832063630" watchObservedRunningTime="2026-09-11 11:49:00.067593535 +0000 UTC m=+71.832566710" Sep 11 11:49:00.084030 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:49:00.083978 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-nq29s" podStartSLOduration=67.297799611 podStartE2EDuration="1m12.083962536s" totalImagesPullingTime="4.786162925s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:47:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:49:00.083619677 +0000 UTC m=+71.848592853" watchObservedRunningTime="2026-09-11 11:49:00.083962536 +0000 UTC m=+71.848935711" Sep 11 11:49:26.111543 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:49:26.111506 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6673d6af-e1b2-443b-83c2-7dc7b653b9d0-metrics-tls\") pod \"dns-default-9f582\" (UID: \"6673d6af-e1b2-443b-83c2-7dc7b653b9d0\") " pod="openshift-dns/dns-default-9f582" Sep 11 11:49:26.112052 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:49:26.111556 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-registry-tls\") pod \"image-registry-9f4c766d4-g58wm\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" Sep 11 11:49:26.112052 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:49:26.111654 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 11 11:49:26.112052 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:49:26.111674 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 11 11:49:26.112052 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:49:26.111685 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9f4c766d4-g58wm: secret "image-registry-tls" not found Sep 11 11:49:26.112052 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:49:26.111737 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6673d6af-e1b2-443b-83c2-7dc7b653b9d0-metrics-tls podName:6673d6af-e1b2-443b-83c2-7dc7b653b9d0 nodeName:}" failed. No retries permitted until 2026-09-11 11:50:30.111716331 +0000 UTC m=+161.876689492 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6673d6af-e1b2-443b-83c2-7dc7b653b9d0-metrics-tls") pod "dns-default-9f582" (UID: "6673d6af-e1b2-443b-83c2-7dc7b653b9d0") : secret "dns-default-metrics-tls" not found Sep 11 11:49:26.112052 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:49:26.111752 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-registry-tls podName:60eec8ae-8c59-40f6-820a-9671b9ec919c nodeName:}" failed. No retries permitted until 2026-09-11 11:50:30.111745648 +0000 UTC m=+161.876718801 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-registry-tls") pod "image-registry-9f4c766d4-g58wm" (UID: "60eec8ae-8c59-40f6-820a-9671b9ec919c") : secret "image-registry-tls" not found Sep 11 11:49:31.053880 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:49:31.053831 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-nq29s" Sep 11 11:49:58.538104 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:49:58.538069 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4fd9da2-13c7-4b85-9c89-f10a28d43049-metrics-certs\") pod \"network-metrics-daemon-x982g\" (UID: \"d4fd9da2-13c7-4b85-9c89-f10a28d43049\") " pod="openshift-multus/network-metrics-daemon-x982g" Sep 11 11:49:58.538546 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:49:58.538190 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 11 11:49:58.538546 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:49:58.538249 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4fd9da2-13c7-4b85-9c89-f10a28d43049-metrics-certs podName:d4fd9da2-13c7-4b85-9c89-f10a28d43049 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:00.538234513 +0000 UTC m=+252.303207665 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4fd9da2-13c7-4b85-9c89-f10a28d43049-metrics-certs") pod "network-metrics-daemon-x982g" (UID: "d4fd9da2-13c7-4b85-9c89-f10a28d43049") : secret "metrics-daemon-secret" not found Sep 11 11:50:09.720796 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:09.720753 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-gncvx"] Sep 11 11:50:09.722780 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:09.722765 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gncvx" Sep 11 11:50:09.728144 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:09.728119 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 11 11:50:09.728366 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:09.728346 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 11 11:50:09.728436 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:09.728417 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 11 11:50:09.728493 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:09.728446 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 11 11:50:09.729198 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:09.729179 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5tl7g\"" Sep 11 11:50:09.740192 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:09.740168 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gncvx"] Sep 11 11:50:09.818802 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:09.818760 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/55e0af20-17e2-45af-a9ae-06d4a206d839-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gncvx\" (UID: \"55e0af20-17e2-45af-a9ae-06d4a206d839\") " pod="openshift-insights/insights-runtime-extractor-gncvx" Sep 11 11:50:09.818802 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:09.818802 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/55e0af20-17e2-45af-a9ae-06d4a206d839-data-volume\") pod \"insights-runtime-extractor-gncvx\" (UID: \"55e0af20-17e2-45af-a9ae-06d4a206d839\") " pod="openshift-insights/insights-runtime-extractor-gncvx" Sep 11 11:50:09.819053 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:09.818824 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/55e0af20-17e2-45af-a9ae-06d4a206d839-crio-socket\") pod \"insights-runtime-extractor-gncvx\" (UID: \"55e0af20-17e2-45af-a9ae-06d4a206d839\") " pod="openshift-insights/insights-runtime-extractor-gncvx" Sep 11 11:50:09.819053 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:09.818965 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/55e0af20-17e2-45af-a9ae-06d4a206d839-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gncvx\" (UID: \"55e0af20-17e2-45af-a9ae-06d4a206d839\") " pod="openshift-insights/insights-runtime-extractor-gncvx" Sep 11 11:50:09.819053 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:09.819001 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvhrj\" (UniqueName: \"kubernetes.io/projected/55e0af20-17e2-45af-a9ae-06d4a206d839-kube-api-access-mvhrj\") pod \"insights-runtime-extractor-gncvx\" (UID: \"55e0af20-17e2-45af-a9ae-06d4a206d839\") " pod="openshift-insights/insights-runtime-extractor-gncvx" Sep 11 11:50:09.919530 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:09.919464 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/55e0af20-17e2-45af-a9ae-06d4a206d839-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gncvx\" (UID: \"55e0af20-17e2-45af-a9ae-06d4a206d839\") " pod="openshift-insights/insights-runtime-extractor-gncvx" Sep 11 11:50:09.919688 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:09.919546 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/55e0af20-17e2-45af-a9ae-06d4a206d839-data-volume\") pod \"insights-runtime-extractor-gncvx\" (UID: \"55e0af20-17e2-45af-a9ae-06d4a206d839\") " pod="openshift-insights/insights-runtime-extractor-gncvx" Sep 11 11:50:09.919688 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:09.919578 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/55e0af20-17e2-45af-a9ae-06d4a206d839-crio-socket\") pod \"insights-runtime-extractor-gncvx\" (UID: \"55e0af20-17e2-45af-a9ae-06d4a206d839\") " pod="openshift-insights/insights-runtime-extractor-gncvx" Sep 11 11:50:09.919688 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:50:09.919598 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 11 11:50:09.919688 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:09.919654 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/55e0af20-17e2-45af-a9ae-06d4a206d839-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gncvx\" (UID: \"55e0af20-17e2-45af-a9ae-06d4a206d839\") " pod="openshift-insights/insights-runtime-extractor-gncvx" Sep 11 11:50:09.919688 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:50:09.919672 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/55e0af20-17e2-45af-a9ae-06d4a206d839-kube-rbac-proxy-cm podName:55e0af20-17e2-45af-a9ae-06d4a206d839 nodeName:}" failed. No retries permitted until 2026-09-11 11:50:10.419652171 +0000 UTC m=+142.184625341 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/55e0af20-17e2-45af-a9ae-06d4a206d839-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gncvx" (UID: "55e0af20-17e2-45af-a9ae-06d4a206d839") : configmap "kube-rbac-proxy" not found Sep 11 11:50:09.919936 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:09.919707 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvhrj\" (UniqueName: \"kubernetes.io/projected/55e0af20-17e2-45af-a9ae-06d4a206d839-kube-api-access-mvhrj\") pod \"insights-runtime-extractor-gncvx\" (UID: \"55e0af20-17e2-45af-a9ae-06d4a206d839\") " pod="openshift-insights/insights-runtime-extractor-gncvx" Sep 11 11:50:09.919936 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:50:09.919743 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 11 11:50:09.919936 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:09.919765 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/55e0af20-17e2-45af-a9ae-06d4a206d839-crio-socket\") pod \"insights-runtime-extractor-gncvx\" (UID: \"55e0af20-17e2-45af-a9ae-06d4a206d839\") " pod="openshift-insights/insights-runtime-extractor-gncvx" Sep 11 11:50:09.919936 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:50:09.919790 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55e0af20-17e2-45af-a9ae-06d4a206d839-insights-runtime-extractor-tls podName:55e0af20-17e2-45af-a9ae-06d4a206d839 nodeName:}" failed. No retries permitted until 2026-09-11 11:50:10.419777722 +0000 UTC m=+142.184750876 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/55e0af20-17e2-45af-a9ae-06d4a206d839-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gncvx" (UID: "55e0af20-17e2-45af-a9ae-06d4a206d839") : secret "insights-runtime-extractor-tls" not found Sep 11 11:50:09.919936 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:09.919918 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/55e0af20-17e2-45af-a9ae-06d4a206d839-data-volume\") pod \"insights-runtime-extractor-gncvx\" (UID: \"55e0af20-17e2-45af-a9ae-06d4a206d839\") " pod="openshift-insights/insights-runtime-extractor-gncvx" Sep 11 11:50:09.936518 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:09.936499 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvhrj\" (UniqueName: \"kubernetes.io/projected/55e0af20-17e2-45af-a9ae-06d4a206d839-kube-api-access-mvhrj\") pod \"insights-runtime-extractor-gncvx\" (UID: \"55e0af20-17e2-45af-a9ae-06d4a206d839\") " pod="openshift-insights/insights-runtime-extractor-gncvx" Sep 11 11:50:10.423884 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:10.423829 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/55e0af20-17e2-45af-a9ae-06d4a206d839-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gncvx\" (UID: \"55e0af20-17e2-45af-a9ae-06d4a206d839\") " pod="openshift-insights/insights-runtime-extractor-gncvx" Sep 11 11:50:10.424075 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:50:10.423956 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 11 11:50:10.424075 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:10.423968 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/55e0af20-17e2-45af-a9ae-06d4a206d839-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gncvx\" (UID: \"55e0af20-17e2-45af-a9ae-06d4a206d839\") " pod="openshift-insights/insights-runtime-extractor-gncvx" Sep 11 11:50:10.424075 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:50:10.424036 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/55e0af20-17e2-45af-a9ae-06d4a206d839-kube-rbac-proxy-cm podName:55e0af20-17e2-45af-a9ae-06d4a206d839 nodeName:}" failed. No retries permitted until 2026-09-11 11:50:11.4240167 +0000 UTC m=+143.188989876 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/55e0af20-17e2-45af-a9ae-06d4a206d839-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gncvx" (UID: "55e0af20-17e2-45af-a9ae-06d4a206d839") : configmap "kube-rbac-proxy" not found Sep 11 11:50:10.424197 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:50:10.424091 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 11 11:50:10.424197 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:50:10.424143 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55e0af20-17e2-45af-a9ae-06d4a206d839-insights-runtime-extractor-tls podName:55e0af20-17e2-45af-a9ae-06d4a206d839 nodeName:}" failed. No retries permitted until 2026-09-11 11:50:11.424130941 +0000 UTC m=+143.189104103 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/55e0af20-17e2-45af-a9ae-06d4a206d839-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gncvx" (UID: "55e0af20-17e2-45af-a9ae-06d4a206d839") : secret "insights-runtime-extractor-tls" not found Sep 11 11:50:11.432833 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:11.432792 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/55e0af20-17e2-45af-a9ae-06d4a206d839-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gncvx\" (UID: \"55e0af20-17e2-45af-a9ae-06d4a206d839\") " pod="openshift-insights/insights-runtime-extractor-gncvx" Sep 11 11:50:11.433227 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:11.432852 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/55e0af20-17e2-45af-a9ae-06d4a206d839-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gncvx\" (UID: \"55e0af20-17e2-45af-a9ae-06d4a206d839\") " pod="openshift-insights/insights-runtime-extractor-gncvx" Sep 11 11:50:11.433227 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:50:11.432947 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 11 11:50:11.433227 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:50:11.432959 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 11 11:50:11.433227 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:50:11.433000 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/55e0af20-17e2-45af-a9ae-06d4a206d839-kube-rbac-proxy-cm podName:55e0af20-17e2-45af-a9ae-06d4a206d839 nodeName:}" failed. No retries permitted until 2026-09-11 11:50:13.432987429 +0000 UTC m=+145.197960582 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/55e0af20-17e2-45af-a9ae-06d4a206d839-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gncvx" (UID: "55e0af20-17e2-45af-a9ae-06d4a206d839") : configmap "kube-rbac-proxy" not found Sep 11 11:50:11.433227 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:50:11.433012 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55e0af20-17e2-45af-a9ae-06d4a206d839-insights-runtime-extractor-tls podName:55e0af20-17e2-45af-a9ae-06d4a206d839 nodeName:}" failed. No retries permitted until 2026-09-11 11:50:13.433007058 +0000 UTC m=+145.197980210 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/55e0af20-17e2-45af-a9ae-06d4a206d839-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gncvx" (UID: "55e0af20-17e2-45af-a9ae-06d4a206d839") : secret "insights-runtime-extractor-tls" not found Sep 11 11:50:12.463647 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:12.463620 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rz4hn_5bf5658e-69eb-4ac8-9359-4653f8fd588f/dns-node-resolver/0.log" Sep 11 11:50:13.263883 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:13.263834 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2sjcg_e0202693-9deb-4f3b-8d1f-3eec2f3f03d6/node-ca/0.log" Sep 11 11:50:13.446120 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:13.446086 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/55e0af20-17e2-45af-a9ae-06d4a206d839-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gncvx\" (UID: \"55e0af20-17e2-45af-a9ae-06d4a206d839\") " pod="openshift-insights/insights-runtime-extractor-gncvx" Sep 11 11:50:13.446244 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:13.446138 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/55e0af20-17e2-45af-a9ae-06d4a206d839-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gncvx\" (UID: \"55e0af20-17e2-45af-a9ae-06d4a206d839\") " pod="openshift-insights/insights-runtime-extractor-gncvx" Sep 11 11:50:13.446244 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:50:13.446204 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 11 11:50:13.446244 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:50:13.446233 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 11 11:50:13.446339 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:50:13.446253 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/55e0af20-17e2-45af-a9ae-06d4a206d839-kube-rbac-proxy-cm podName:55e0af20-17e2-45af-a9ae-06d4a206d839 nodeName:}" failed. No retries permitted until 2026-09-11 11:50:17.44624177 +0000 UTC m=+149.211214923 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/55e0af20-17e2-45af-a9ae-06d4a206d839-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gncvx" (UID: "55e0af20-17e2-45af-a9ae-06d4a206d839") : configmap "kube-rbac-proxy" not found Sep 11 11:50:13.446339 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:50:13.446310 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55e0af20-17e2-45af-a9ae-06d4a206d839-insights-runtime-extractor-tls podName:55e0af20-17e2-45af-a9ae-06d4a206d839 nodeName:}" failed. No retries permitted until 2026-09-11 11:50:17.446296545 +0000 UTC m=+149.211269698 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/55e0af20-17e2-45af-a9ae-06d4a206d839-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gncvx" (UID: "55e0af20-17e2-45af-a9ae-06d4a206d839") : secret "insights-runtime-extractor-tls" not found Sep 11 11:50:14.074714 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:14.074683 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5qrzc_d1190be0-add2-48ea-ad87-5d7271697bfa/serve-healthcheck-canary/0.log" Sep 11 11:50:17.480809 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:17.480770 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/55e0af20-17e2-45af-a9ae-06d4a206d839-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gncvx\" (UID: \"55e0af20-17e2-45af-a9ae-06d4a206d839\") " pod="openshift-insights/insights-runtime-extractor-gncvx" Sep 11 11:50:17.481238 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:17.480893 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/55e0af20-17e2-45af-a9ae-06d4a206d839-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gncvx\" (UID: \"55e0af20-17e2-45af-a9ae-06d4a206d839\") " pod="openshift-insights/insights-runtime-extractor-gncvx" Sep 11 11:50:17.481238 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:50:17.480917 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 11 11:50:17.481238 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:50:17.480997 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/55e0af20-17e2-45af-a9ae-06d4a206d839-kube-rbac-proxy-cm podName:55e0af20-17e2-45af-a9ae-06d4a206d839 nodeName:}" failed. No retries permitted until 2026-09-11 11:50:25.480979845 +0000 UTC m=+157.245953020 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/55e0af20-17e2-45af-a9ae-06d4a206d839-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gncvx" (UID: "55e0af20-17e2-45af-a9ae-06d4a206d839") : configmap "kube-rbac-proxy" not found Sep 11 11:50:17.481238 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:50:17.480999 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 11 11:50:17.481238 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:50:17.481034 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55e0af20-17e2-45af-a9ae-06d4a206d839-insights-runtime-extractor-tls podName:55e0af20-17e2-45af-a9ae-06d4a206d839 nodeName:}" failed. No retries permitted until 2026-09-11 11:50:25.481024063 +0000 UTC m=+157.245997230 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/55e0af20-17e2-45af-a9ae-06d4a206d839-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gncvx" (UID: "55e0af20-17e2-45af-a9ae-06d4a206d839") : secret "insights-runtime-extractor-tls" not found Sep 11 11:50:25.173332 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:50:25.173284 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" podUID="60eec8ae-8c59-40f6-820a-9671b9ec919c" Sep 11 11:50:25.192718 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:50:25.192684 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-9f582" podUID="6673d6af-e1b2-443b-83c2-7dc7b653b9d0" Sep 11 11:50:25.290182 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:25.290153 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9f582" Sep 11 11:50:25.290324 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:25.290153 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" Sep 11 11:50:25.550169 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:25.550073 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/55e0af20-17e2-45af-a9ae-06d4a206d839-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gncvx\" (UID: \"55e0af20-17e2-45af-a9ae-06d4a206d839\") " pod="openshift-insights/insights-runtime-extractor-gncvx" Sep 11 11:50:25.550169 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:25.550135 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/55e0af20-17e2-45af-a9ae-06d4a206d839-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gncvx\" (UID: \"55e0af20-17e2-45af-a9ae-06d4a206d839\") " pod="openshift-insights/insights-runtime-extractor-gncvx" Sep 11 11:50:25.550353 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:50:25.550231 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 11 11:50:25.550353 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:50:25.550309 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55e0af20-17e2-45af-a9ae-06d4a206d839-insights-runtime-extractor-tls podName:55e0af20-17e2-45af-a9ae-06d4a206d839 nodeName:}" failed. No retries permitted until 2026-09-11 11:50:41.550291328 +0000 UTC m=+173.315264495 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/55e0af20-17e2-45af-a9ae-06d4a206d839-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gncvx" (UID: "55e0af20-17e2-45af-a9ae-06d4a206d839") : secret "insights-runtime-extractor-tls" not found Sep 11 11:50:25.550556 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:25.550540 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/55e0af20-17e2-45af-a9ae-06d4a206d839-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gncvx\" (UID: \"55e0af20-17e2-45af-a9ae-06d4a206d839\") " pod="openshift-insights/insights-runtime-extractor-gncvx" Sep 11 11:50:25.781668 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:50:25.781628 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-x982g" podUID="d4fd9da2-13c7-4b85-9c89-f10a28d43049" Sep 11 11:50:30.188524 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:30.188482 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6673d6af-e1b2-443b-83c2-7dc7b653b9d0-metrics-tls\") pod \"dns-default-9f582\" (UID: \"6673d6af-e1b2-443b-83c2-7dc7b653b9d0\") " pod="openshift-dns/dns-default-9f582" Sep 11 11:50:30.189007 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:30.188535 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-registry-tls\") pod \"image-registry-9f4c766d4-g58wm\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" Sep 11 11:50:30.190877 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:30.190835 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6673d6af-e1b2-443b-83c2-7dc7b653b9d0-metrics-tls\") pod \"dns-default-9f582\" (UID: \"6673d6af-e1b2-443b-83c2-7dc7b653b9d0\") " pod="openshift-dns/dns-default-9f582" Sep 11 11:50:30.190983 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:30.190960 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-registry-tls\") pod \"image-registry-9f4c766d4-g58wm\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" Sep 11 11:50:30.395025 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:30.394988 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-47c8b\"" Sep 11 11:50:30.395025 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:30.394999 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-m9sw5\"" Sep 11 11:50:30.401874 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:30.401844 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9f582" Sep 11 11:50:30.401955 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:30.401922 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" Sep 11 11:50:30.683127 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:30.683096 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9f582"] Sep 11 11:50:30.685958 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:30.685930 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-9f4c766d4-g58wm"] Sep 11 11:50:30.686886 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:50:30.686848 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6673d6af_e1b2_443b_83c2_7dc7b653b9d0.slice/crio-294e7508c021f6c9beda1e2a4e98e429dae990c4dbff383c5ee59a406e2cf0b5 WatchSource:0}: Error finding container 294e7508c021f6c9beda1e2a4e98e429dae990c4dbff383c5ee59a406e2cf0b5: Status 404 returned error can't find the container with id 294e7508c021f6c9beda1e2a4e98e429dae990c4dbff383c5ee59a406e2cf0b5 Sep 11 11:50:30.688693 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:50:30.688670 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60eec8ae_8c59_40f6_820a_9671b9ec919c.slice/crio-f286e3c7c191b26bf4c5eb6bda3f9e80249399bfd98138875d7c236222ceea4d WatchSource:0}: Error finding container f286e3c7c191b26bf4c5eb6bda3f9e80249399bfd98138875d7c236222ceea4d: Status 404 returned error can't find the container with id f286e3c7c191b26bf4c5eb6bda3f9e80249399bfd98138875d7c236222ceea4d Sep 11 11:50:31.309170 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:31.309126 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9f582" event={"ID":"6673d6af-e1b2-443b-83c2-7dc7b653b9d0","Type":"ContainerStarted","Data":"294e7508c021f6c9beda1e2a4e98e429dae990c4dbff383c5ee59a406e2cf0b5"} Sep 11 11:50:31.310579 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:31.310555 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" event={"ID":"60eec8ae-8c59-40f6-820a-9671b9ec919c","Type":"ContainerStarted","Data":"a03076e01344bff355c4e50352cc3d84c5b121476efcc31d42bcd546a6ed6a7e"} Sep 11 11:50:31.310670 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:31.310595 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" event={"ID":"60eec8ae-8c59-40f6-820a-9671b9ec919c","Type":"ContainerStarted","Data":"f286e3c7c191b26bf4c5eb6bda3f9e80249399bfd98138875d7c236222ceea4d"} Sep 11 11:50:31.310670 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:31.310636 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" Sep 11 11:50:31.337034 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:31.336989 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" podStartSLOduration=162.336975306 podStartE2EDuration="2m42.336975306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:47:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:50:31.335414294 +0000 UTC m=+163.100387469" watchObservedRunningTime="2026-09-11 11:50:31.336975306 +0000 UTC m=+163.101948481" Sep 11 11:50:33.317939 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:33.317908 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9f582" event={"ID":"6673d6af-e1b2-443b-83c2-7dc7b653b9d0","Type":"ContainerStarted","Data":"c3d2fab5792be48baac10e21741fdf31d3d8bc9a89ee1005972be2b56bc53e53"} Sep 11 11:50:33.318325 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:33.317964 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9f582" event={"ID":"6673d6af-e1b2-443b-83c2-7dc7b653b9d0","Type":"ContainerStarted","Data":"3be10dfb9da6ea414a52194810a10b65498bc50332681f8ef117391ceebfeeb6"} Sep 11 11:50:33.318325 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:33.318049 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9f582" Sep 11 11:50:36.972642 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:36.972584 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d44464d58-zfgcj" podUID="d1d3ab73-3bf4-494e-bd65-b92790da8f33" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.10:8000/readyz\": dial tcp 10.132.0.10:8000: connect: connection refused" Sep 11 11:50:37.330705 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:37.330674 2566 generic.go:356] "Generic (PLEG): container finished" podID="60572541-4256-4131-9831-50ef928135e1" containerID="b1c46143990ccd73b7a0874a8d7d2e0b24b0817c8a7ab4da2c0a43e29d73ff92" exitCode=255 Sep 11 11:50:37.330898 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:37.330759 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84cbf47796-f5lw9" event={"ID":"60572541-4256-4131-9831-50ef928135e1","Type":"ContainerDied","Data":"b1c46143990ccd73b7a0874a8d7d2e0b24b0817c8a7ab4da2c0a43e29d73ff92"} Sep 11 11:50:37.331138 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:37.331123 2566 scope.go:117] "RemoveContainer" containerID="b1c46143990ccd73b7a0874a8d7d2e0b24b0817c8a7ab4da2c0a43e29d73ff92" Sep 11 11:50:37.332402 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:37.332382 2566 generic.go:356] "Generic (PLEG): container finished" podID="d1d3ab73-3bf4-494e-bd65-b92790da8f33" containerID="a23de5b596bcd30cc69af66df79e38a88df42619111531b07904f3047895b296" exitCode=1 Sep 11 11:50:37.332467 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:37.332431 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d44464d58-zfgcj" event={"ID":"d1d3ab73-3bf4-494e-bd65-b92790da8f33","Type":"ContainerDied","Data":"a23de5b596bcd30cc69af66df79e38a88df42619111531b07904f3047895b296"} Sep 11 11:50:37.332730 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:37.332717 2566 scope.go:117] "RemoveContainer" containerID="a23de5b596bcd30cc69af66df79e38a88df42619111531b07904f3047895b296" Sep 11 11:50:37.352131 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:37.352068 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9f582" podStartSLOduration=133.15158009 podStartE2EDuration="2m15.348832463s" totalImagesPullingTime="2.197252373s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:48:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:50:33.340322892 +0000 UTC m=+165.105296067" watchObservedRunningTime="2026-09-11 11:50:37.348832463 +0000 UTC m=+169.113805641" Sep 11 11:50:38.337705 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:38.337661 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d44464d58-zfgcj" event={"ID":"d1d3ab73-3bf4-494e-bd65-b92790da8f33","Type":"ContainerStarted","Data":"f8bfa26b051ea07da66b5c4a5ac7dd5aa8b7969376edfbb8aa8ac3ba810e0b52"} Sep 11 11:50:38.338178 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:38.338069 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d44464d58-zfgcj" Sep 11 11:50:38.338714 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:38.338692 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d44464d58-zfgcj" Sep 11 11:50:38.339494 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:38.339476 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84cbf47796-f5lw9" event={"ID":"60572541-4256-4131-9831-50ef928135e1","Type":"ContainerStarted","Data":"d2d41255b858fbeb6cf2c916db1205327327dd50650de3898314b356a57a3191"} Sep 11 11:50:38.753595 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:38.753519 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x982g" Sep 11 11:50:41.579427 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:41.579313 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/55e0af20-17e2-45af-a9ae-06d4a206d839-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gncvx\" (UID: \"55e0af20-17e2-45af-a9ae-06d4a206d839\") " pod="openshift-insights/insights-runtime-extractor-gncvx" Sep 11 11:50:41.581605 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:41.581581 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/55e0af20-17e2-45af-a9ae-06d4a206d839-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gncvx\" (UID: \"55e0af20-17e2-45af-a9ae-06d4a206d839\") " pod="openshift-insights/insights-runtime-extractor-gncvx" Sep 11 11:50:41.832088 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:41.831999 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gncvx" Sep 11 11:50:42.102800 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:42.102704 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gncvx"] Sep 11 11:50:42.106615 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:50:42.106581 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55e0af20_17e2_45af_a9ae_06d4a206d839.slice/crio-54ef5f472629d023f666be411df19008ef8a0f4a85a13f4cc7628596c7f409d1 WatchSource:0}: Error finding container 54ef5f472629d023f666be411df19008ef8a0f4a85a13f4cc7628596c7f409d1: Status 404 returned error can't find the container with id 54ef5f472629d023f666be411df19008ef8a0f4a85a13f4cc7628596c7f409d1 Sep 11 11:50:42.355221 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:42.355131 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gncvx" event={"ID":"55e0af20-17e2-45af-a9ae-06d4a206d839","Type":"ContainerStarted","Data":"ebac952630f3c3f61a127216e77b321b63b877586ebdb115c9972667c8514c85"} Sep 11 11:50:42.355221 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:42.355192 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gncvx" event={"ID":"55e0af20-17e2-45af-a9ae-06d4a206d839","Type":"ContainerStarted","Data":"54ef5f472629d023f666be411df19008ef8a0f4a85a13f4cc7628596c7f409d1"} Sep 11 11:50:43.323461 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:43.323432 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9f582" Sep 11 11:50:43.359682 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:43.359646 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gncvx" event={"ID":"55e0af20-17e2-45af-a9ae-06d4a206d839","Type":"ContainerStarted","Data":"40ab7c844791c263c312a20d1c3e2f62acc2dfa9665b1b8277b29ad5c2c0c8eb"} Sep 11 11:50:45.368103 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:45.368070 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gncvx" event={"ID":"55e0af20-17e2-45af-a9ae-06d4a206d839","Type":"ContainerStarted","Data":"fcedcbee7fb0ad593534f38ff6c9e5ba3a1d7a4c1df7802fc01408505bec37aa"} Sep 11 11:50:45.388131 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:45.388081 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-gncvx" podStartSLOduration=34.048585785 podStartE2EDuration="36.388065262s" totalImagesPullingTime="2.339479477s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:50:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:50:45.387633916 +0000 UTC m=+177.152607095" watchObservedRunningTime="2026-09-11 11:50:45.388065262 +0000 UTC m=+177.153038437" Sep 11 11:50:47.755298 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:47.755265 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-48knj"] Sep 11 11:50:47.758292 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:47.758275 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-48knj" Sep 11 11:50:47.763915 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:47.763886 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 11 11:50:47.764046 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:47.763929 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 11 11:50:47.764046 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:47.763959 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 11 11:50:47.764046 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:47.763890 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 11 11:50:47.764731 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:47.764711 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-lh48f\"" Sep 11 11:50:47.764832 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:47.764815 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 11 11:50:47.764938 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:47.764922 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 11 11:50:47.824614 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:47.824579 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/45cf0550-9900-445d-8813-ff3881e3fec9-root\") pod \"node-exporter-48knj\" (UID: \"45cf0550-9900-445d-8813-ff3881e3fec9\") " pod="openshift-monitoring/node-exporter-48knj" Sep 11 11:50:47.824614 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:47.824617 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/45cf0550-9900-445d-8813-ff3881e3fec9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-48knj\" (UID: \"45cf0550-9900-445d-8813-ff3881e3fec9\") " pod="openshift-monitoring/node-exporter-48knj" Sep 11 11:50:47.824891 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:47.824650 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/45cf0550-9900-445d-8813-ff3881e3fec9-node-exporter-tls\") pod \"node-exporter-48knj\" (UID: \"45cf0550-9900-445d-8813-ff3881e3fec9\") " pod="openshift-monitoring/node-exporter-48knj" Sep 11 11:50:47.824891 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:47.824669 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/45cf0550-9900-445d-8813-ff3881e3fec9-node-exporter-wtmp\") pod \"node-exporter-48knj\" (UID: \"45cf0550-9900-445d-8813-ff3881e3fec9\") " pod="openshift-monitoring/node-exporter-48knj" Sep 11 11:50:47.824891 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:47.824719 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/45cf0550-9900-445d-8813-ff3881e3fec9-sys\") pod \"node-exporter-48knj\" (UID: \"45cf0550-9900-445d-8813-ff3881e3fec9\") " pod="openshift-monitoring/node-exporter-48knj" Sep 11 11:50:47.824891 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:47.824750 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wxxw\" (UniqueName: \"kubernetes.io/projected/45cf0550-9900-445d-8813-ff3881e3fec9-kube-api-access-6wxxw\") pod \"node-exporter-48knj\" (UID: \"45cf0550-9900-445d-8813-ff3881e3fec9\") " pod="openshift-monitoring/node-exporter-48knj" Sep 11 11:50:47.824891 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:47.824780 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45cf0550-9900-445d-8813-ff3881e3fec9-metrics-client-ca\") pod \"node-exporter-48knj\" (UID: \"45cf0550-9900-445d-8813-ff3881e3fec9\") " pod="openshift-monitoring/node-exporter-48knj" Sep 11 11:50:47.824891 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:47.824819 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/45cf0550-9900-445d-8813-ff3881e3fec9-node-exporter-accelerators-collector-config\") pod \"node-exporter-48knj\" (UID: \"45cf0550-9900-445d-8813-ff3881e3fec9\") " pod="openshift-monitoring/node-exporter-48knj" Sep 11 11:50:47.824891 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:47.824846 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/45cf0550-9900-445d-8813-ff3881e3fec9-node-exporter-textfile\") pod \"node-exporter-48knj\" (UID: \"45cf0550-9900-445d-8813-ff3881e3fec9\") " pod="openshift-monitoring/node-exporter-48knj" Sep 11 11:50:47.926015 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:47.925978 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/45cf0550-9900-445d-8813-ff3881e3fec9-sys\") pod \"node-exporter-48knj\" (UID: \"45cf0550-9900-445d-8813-ff3881e3fec9\") " pod="openshift-monitoring/node-exporter-48knj" Sep 11 11:50:47.926015 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:47.926011 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wxxw\" (UniqueName: \"kubernetes.io/projected/45cf0550-9900-445d-8813-ff3881e3fec9-kube-api-access-6wxxw\") pod \"node-exporter-48knj\" (UID: \"45cf0550-9900-445d-8813-ff3881e3fec9\") " pod="openshift-monitoring/node-exporter-48knj" Sep 11 11:50:47.926260 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:47.926038 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45cf0550-9900-445d-8813-ff3881e3fec9-metrics-client-ca\") pod \"node-exporter-48knj\" (UID: \"45cf0550-9900-445d-8813-ff3881e3fec9\") " pod="openshift-monitoring/node-exporter-48knj" Sep 11 11:50:47.926260 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:47.926061 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/45cf0550-9900-445d-8813-ff3881e3fec9-node-exporter-accelerators-collector-config\") pod \"node-exporter-48knj\" (UID: \"45cf0550-9900-445d-8813-ff3881e3fec9\") " pod="openshift-monitoring/node-exporter-48knj" Sep 11 11:50:47.926260 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:47.926084 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/45cf0550-9900-445d-8813-ff3881e3fec9-node-exporter-textfile\") pod \"node-exporter-48knj\" (UID: \"45cf0550-9900-445d-8813-ff3881e3fec9\") " pod="openshift-monitoring/node-exporter-48knj" Sep 11 11:50:47.926260 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:47.926115 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/45cf0550-9900-445d-8813-ff3881e3fec9-root\") pod \"node-exporter-48knj\" (UID: \"45cf0550-9900-445d-8813-ff3881e3fec9\") " pod="openshift-monitoring/node-exporter-48knj" Sep 11 11:50:47.926260 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:47.926107 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/45cf0550-9900-445d-8813-ff3881e3fec9-sys\") pod \"node-exporter-48knj\" (UID: \"45cf0550-9900-445d-8813-ff3881e3fec9\") " pod="openshift-monitoring/node-exporter-48knj" Sep 11 11:50:47.926260 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:47.926185 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/45cf0550-9900-445d-8813-ff3881e3fec9-root\") pod \"node-exporter-48knj\" (UID: \"45cf0550-9900-445d-8813-ff3881e3fec9\") " pod="openshift-monitoring/node-exporter-48knj" Sep 11 11:50:47.926260 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:47.926201 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/45cf0550-9900-445d-8813-ff3881e3fec9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-48knj\" (UID: \"45cf0550-9900-445d-8813-ff3881e3fec9\") " pod="openshift-monitoring/node-exporter-48knj" Sep 11 11:50:47.926260 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:47.926248 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/45cf0550-9900-445d-8813-ff3881e3fec9-node-exporter-tls\") pod \"node-exporter-48knj\" (UID: \"45cf0550-9900-445d-8813-ff3881e3fec9\") " pod="openshift-monitoring/node-exporter-48knj" Sep 11 11:50:47.926641 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:47.926280 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/45cf0550-9900-445d-8813-ff3881e3fec9-node-exporter-wtmp\") pod \"node-exporter-48knj\" (UID: \"45cf0550-9900-445d-8813-ff3881e3fec9\") " pod="openshift-monitoring/node-exporter-48knj" Sep 11 11:50:47.926641 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:47.926418 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/45cf0550-9900-445d-8813-ff3881e3fec9-node-exporter-wtmp\") pod \"node-exporter-48knj\" (UID: \"45cf0550-9900-445d-8813-ff3881e3fec9\") " pod="openshift-monitoring/node-exporter-48knj" Sep 11 11:50:47.926641 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:47.926494 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/45cf0550-9900-445d-8813-ff3881e3fec9-node-exporter-textfile\") pod \"node-exporter-48knj\" (UID: \"45cf0550-9900-445d-8813-ff3881e3fec9\") " pod="openshift-monitoring/node-exporter-48knj" Sep 11 11:50:47.926794 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:47.926698 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45cf0550-9900-445d-8813-ff3881e3fec9-metrics-client-ca\") pod \"node-exporter-48knj\" (UID: \"45cf0550-9900-445d-8813-ff3881e3fec9\") " pod="openshift-monitoring/node-exporter-48knj" Sep 11 11:50:47.926794 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:47.926748 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/45cf0550-9900-445d-8813-ff3881e3fec9-node-exporter-accelerators-collector-config\") pod \"node-exporter-48knj\" (UID: \"45cf0550-9900-445d-8813-ff3881e3fec9\") " pod="openshift-monitoring/node-exporter-48knj" Sep 11 11:50:47.928592 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:47.928571 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/45cf0550-9900-445d-8813-ff3881e3fec9-node-exporter-tls\") pod \"node-exporter-48knj\" (UID: \"45cf0550-9900-445d-8813-ff3881e3fec9\") " pod="openshift-monitoring/node-exporter-48knj" Sep 11 11:50:47.928695 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:47.928624 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/45cf0550-9900-445d-8813-ff3881e3fec9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-48knj\" (UID: \"45cf0550-9900-445d-8813-ff3881e3fec9\") " pod="openshift-monitoring/node-exporter-48knj" Sep 11 11:50:47.934192 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:47.934170 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wxxw\" (UniqueName: \"kubernetes.io/projected/45cf0550-9900-445d-8813-ff3881e3fec9-kube-api-access-6wxxw\") pod \"node-exporter-48knj\" (UID: \"45cf0550-9900-445d-8813-ff3881e3fec9\") " pod="openshift-monitoring/node-exporter-48knj" Sep 11 11:50:48.067554 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:48.067520 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-48knj" Sep 11 11:50:48.078470 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:50:48.078445 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45cf0550_9900_445d_8813_ff3881e3fec9.slice/crio-bc66b5009f059a773cc138562d9a2d2b842f54d5b98e707ce786e366adf0f2d4 WatchSource:0}: Error finding container bc66b5009f059a773cc138562d9a2d2b842f54d5b98e707ce786e366adf0f2d4: Status 404 returned error can't find the container with id bc66b5009f059a773cc138562d9a2d2b842f54d5b98e707ce786e366adf0f2d4 Sep 11 11:50:48.379017 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:48.378912 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-48knj" event={"ID":"45cf0550-9900-445d-8813-ff3881e3fec9","Type":"ContainerStarted","Data":"bc66b5009f059a773cc138562d9a2d2b842f54d5b98e707ce786e366adf0f2d4"} Sep 11 11:50:50.406685 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:50.406647 2566 patch_prober.go:28] interesting pod/image-registry-9f4c766d4-g58wm container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 11 11:50:50.407052 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:50.406709 2566 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" podUID="60eec8ae-8c59-40f6-820a-9671b9ec919c" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 11:50:51.389721 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:51.389676 2566 generic.go:356] "Generic (PLEG): container finished" podID="45cf0550-9900-445d-8813-ff3881e3fec9" containerID="829e429f058fd804064f4c005ce46ca2c513860783400b90e4ddbf46dc2f4389" exitCode=0 Sep 11 11:50:51.389721 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:51.389722 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-48knj" event={"ID":"45cf0550-9900-445d-8813-ff3881e3fec9","Type":"ContainerDied","Data":"829e429f058fd804064f4c005ce46ca2c513860783400b90e4ddbf46dc2f4389"} Sep 11 11:50:52.317847 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:52.317811 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" Sep 11 11:50:52.395664 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:52.395627 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-48knj" event={"ID":"45cf0550-9900-445d-8813-ff3881e3fec9","Type":"ContainerStarted","Data":"47d18d89fa5fde2942df40f9c3a209774f7166a92cf1ecb5afe75a0b5bff344d"} Sep 11 11:50:52.395814 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:52.395697 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-48knj" event={"ID":"45cf0550-9900-445d-8813-ff3881e3fec9","Type":"ContainerStarted","Data":"802c6722486606cb70f263fcea5f3820ec92b0349188f51de1dcd2a92fda7295"} Sep 11 11:50:52.415534 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:52.415488 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-48knj" podStartSLOduration=3.098339247 podStartE2EDuration="5.415472129s" totalImagesPullingTime="2.256347144s" totalInitContainerRuntime="60.785738ms" isStatefulPod=true podCreationTimestamp="2026-09-11 11:50:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:50:52.414789653 +0000 UTC m=+184.179762825" watchObservedRunningTime="2026-09-11 11:50:52.415472129 +0000 UTC m=+184.180445303" Sep 11 11:50:58.669712 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:50:58.669676 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-9f4c766d4-g58wm"] Sep 11 11:51:18.576805 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:18.576759 2566 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6467cf759d-jvf6s" podUID="60ce193d-c29c-41eb-a8f2-e0148c0584ef" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Sep 11 11:51:23.687824 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:23.687763 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" podUID="60eec8ae-8c59-40f6-820a-9671b9ec919c" containerName="registry" containerID="cri-o://a03076e01344bff355c4e50352cc3d84c5b121476efcc31d42bcd546a6ed6a7e" gracePeriod=30 Sep 11 11:51:23.926646 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:23.926620 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" Sep 11 11:51:24.013949 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:24.013843 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-registry-tls\") pod \"60eec8ae-8c59-40f6-820a-9671b9ec919c\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " Sep 11 11:51:24.013949 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:24.013909 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/60eec8ae-8c59-40f6-820a-9671b9ec919c-image-registry-private-configuration\") pod \"60eec8ae-8c59-40f6-820a-9671b9ec919c\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " Sep 11 11:51:24.014170 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:24.013951 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/60eec8ae-8c59-40f6-820a-9671b9ec919c-ca-trust-extracted\") pod \"60eec8ae-8c59-40f6-820a-9671b9ec919c\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " Sep 11 11:51:24.014170 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:24.013981 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/60eec8ae-8c59-40f6-820a-9671b9ec919c-installation-pull-secrets\") pod \"60eec8ae-8c59-40f6-820a-9671b9ec919c\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " Sep 11 11:51:24.014170 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:24.014006 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-bound-sa-token\") pod \"60eec8ae-8c59-40f6-820a-9671b9ec919c\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " Sep 11 11:51:24.014170 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:24.014027 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/60eec8ae-8c59-40f6-820a-9671b9ec919c-registry-certificates\") pod \"60eec8ae-8c59-40f6-820a-9671b9ec919c\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " Sep 11 11:51:24.014170 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:24.014065 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nknpl\" (UniqueName: \"kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-kube-api-access-nknpl\") pod \"60eec8ae-8c59-40f6-820a-9671b9ec919c\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " Sep 11 11:51:24.014170 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:24.014085 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/60eec8ae-8c59-40f6-820a-9671b9ec919c-trusted-ca\") pod \"60eec8ae-8c59-40f6-820a-9671b9ec919c\" (UID: \"60eec8ae-8c59-40f6-820a-9671b9ec919c\") " Sep 11 11:51:24.014705 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:24.014671 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60eec8ae-8c59-40f6-820a-9671b9ec919c-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "60eec8ae-8c59-40f6-820a-9671b9ec919c" (UID: "60eec8ae-8c59-40f6-820a-9671b9ec919c"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 11:51:24.014843 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:24.014798 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60eec8ae-8c59-40f6-820a-9671b9ec919c-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "60eec8ae-8c59-40f6-820a-9671b9ec919c" (UID: "60eec8ae-8c59-40f6-820a-9671b9ec919c"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 11:51:24.016365 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:24.016337 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "60eec8ae-8c59-40f6-820a-9671b9ec919c" (UID: "60eec8ae-8c59-40f6-820a-9671b9ec919c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 11:51:24.016462 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:24.016412 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60eec8ae-8c59-40f6-820a-9671b9ec919c-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "60eec8ae-8c59-40f6-820a-9671b9ec919c" (UID: "60eec8ae-8c59-40f6-820a-9671b9ec919c"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 11:51:24.016582 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:24.016561 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "60eec8ae-8c59-40f6-820a-9671b9ec919c" (UID: "60eec8ae-8c59-40f6-820a-9671b9ec919c"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 11:51:24.016705 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:24.016685 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-kube-api-access-nknpl" (OuterVolumeSpecName: "kube-api-access-nknpl") pod "60eec8ae-8c59-40f6-820a-9671b9ec919c" (UID: "60eec8ae-8c59-40f6-820a-9671b9ec919c"). InnerVolumeSpecName "kube-api-access-nknpl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 11:51:24.016750 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:24.016710 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60eec8ae-8c59-40f6-820a-9671b9ec919c-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "60eec8ae-8c59-40f6-820a-9671b9ec919c" (UID: "60eec8ae-8c59-40f6-820a-9671b9ec919c"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 11:51:24.022679 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:24.022653 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60eec8ae-8c59-40f6-820a-9671b9ec919c-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "60eec8ae-8c59-40f6-820a-9671b9ec919c" (UID: "60eec8ae-8c59-40f6-820a-9671b9ec919c"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 11:51:24.114966 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:24.114917 2566 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-bound-sa-token\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 11:51:24.114966 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:24.114959 2566 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/60eec8ae-8c59-40f6-820a-9671b9ec919c-registry-certificates\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 11:51:24.114966 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:24.114972 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nknpl\" (UniqueName: \"kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-kube-api-access-nknpl\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 11:51:24.114966 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:24.114981 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/60eec8ae-8c59-40f6-820a-9671b9ec919c-trusted-ca\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 11:51:24.114966 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:24.114989 2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/60eec8ae-8c59-40f6-820a-9671b9ec919c-registry-tls\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 11:51:24.115295 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:24.114998 2566 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/60eec8ae-8c59-40f6-820a-9671b9ec919c-image-registry-private-configuration\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 11:51:24.115295 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:24.115007 2566 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/60eec8ae-8c59-40f6-820a-9671b9ec919c-ca-trust-extracted\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 11:51:24.115295 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:24.115015 2566 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/60eec8ae-8c59-40f6-820a-9671b9ec919c-installation-pull-secrets\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 11:51:24.497276 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:24.497236 2566 generic.go:356] "Generic (PLEG): container finished" podID="60eec8ae-8c59-40f6-820a-9671b9ec919c" containerID="a03076e01344bff355c4e50352cc3d84c5b121476efcc31d42bcd546a6ed6a7e" exitCode=0 Sep 11 11:51:24.497442 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:24.497283 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" Sep 11 11:51:24.497442 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:24.497303 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" event={"ID":"60eec8ae-8c59-40f6-820a-9671b9ec919c","Type":"ContainerDied","Data":"a03076e01344bff355c4e50352cc3d84c5b121476efcc31d42bcd546a6ed6a7e"} Sep 11 11:51:24.497442 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:24.497349 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9f4c766d4-g58wm" event={"ID":"60eec8ae-8c59-40f6-820a-9671b9ec919c","Type":"ContainerDied","Data":"f286e3c7c191b26bf4c5eb6bda3f9e80249399bfd98138875d7c236222ceea4d"} Sep 11 11:51:24.497442 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:24.497383 2566 scope.go:117] "RemoveContainer" containerID="a03076e01344bff355c4e50352cc3d84c5b121476efcc31d42bcd546a6ed6a7e" Sep 11 11:51:24.505359 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:24.505340 2566 scope.go:117] "RemoveContainer" containerID="a03076e01344bff355c4e50352cc3d84c5b121476efcc31d42bcd546a6ed6a7e" Sep 11 11:51:24.505633 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:51:24.505615 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a03076e01344bff355c4e50352cc3d84c5b121476efcc31d42bcd546a6ed6a7e\": container with ID starting with a03076e01344bff355c4e50352cc3d84c5b121476efcc31d42bcd546a6ed6a7e not found: ID does not exist" containerID="a03076e01344bff355c4e50352cc3d84c5b121476efcc31d42bcd546a6ed6a7e" Sep 11 11:51:24.505677 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:24.505640 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a03076e01344bff355c4e50352cc3d84c5b121476efcc31d42bcd546a6ed6a7e"} err="failed to get container status \"a03076e01344bff355c4e50352cc3d84c5b121476efcc31d42bcd546a6ed6a7e\": rpc error: code = NotFound desc = could not find container \"a03076e01344bff355c4e50352cc3d84c5b121476efcc31d42bcd546a6ed6a7e\": container with ID starting with a03076e01344bff355c4e50352cc3d84c5b121476efcc31d42bcd546a6ed6a7e not found: ID does not exist" Sep 11 11:51:24.523913 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:24.523855 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-9f4c766d4-g58wm"] Sep 11 11:51:24.525262 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:24.525217 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-9f4c766d4-g58wm"] Sep 11 11:51:24.757388 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:24.757307 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60eec8ae-8c59-40f6-820a-9671b9ec919c" path="/var/lib/kubelet/pods/60eec8ae-8c59-40f6-820a-9671b9ec919c/volumes" Sep 11 11:51:28.576462 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:28.576425 2566 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6467cf759d-jvf6s" podUID="60ce193d-c29c-41eb-a8f2-e0148c0584ef" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Sep 11 11:51:38.577586 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:38.577545 2566 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6467cf759d-jvf6s" podUID="60ce193d-c29c-41eb-a8f2-e0148c0584ef" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Sep 11 11:51:38.578065 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:38.577635 2566 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6467cf759d-jvf6s" Sep 11 11:51:38.578242 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:38.578220 2566 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"9bb80a01725f4860b98ef4a90e8c1a06139605bf9c5f256816a16c0ffbd1d9e5"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6467cf759d-jvf6s" containerMessage="Container service-proxy failed liveness probe, will be restarted" Sep 11 11:51:38.578322 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:38.578268 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6467cf759d-jvf6s" podUID="60ce193d-c29c-41eb-a8f2-e0148c0584ef" containerName="service-proxy" containerID="cri-o://9bb80a01725f4860b98ef4a90e8c1a06139605bf9c5f256816a16c0ffbd1d9e5" gracePeriod=30 Sep 11 11:51:39.548424 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:39.548390 2566 generic.go:356] "Generic (PLEG): container finished" podID="60ce193d-c29c-41eb-a8f2-e0148c0584ef" containerID="9bb80a01725f4860b98ef4a90e8c1a06139605bf9c5f256816a16c0ffbd1d9e5" exitCode=2 Sep 11 11:51:39.548591 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:39.548463 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6467cf759d-jvf6s" event={"ID":"60ce193d-c29c-41eb-a8f2-e0148c0584ef","Type":"ContainerDied","Data":"9bb80a01725f4860b98ef4a90e8c1a06139605bf9c5f256816a16c0ffbd1d9e5"} Sep 11 11:51:39.548591 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:51:39.548508 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6467cf759d-jvf6s" event={"ID":"60ce193d-c29c-41eb-a8f2-e0148c0584ef","Type":"ContainerStarted","Data":"5277cc8969cfa75b473b27bf36c5d12d9dde0b38251d752250cf1ff98f40f301"} Sep 11 11:52:00.565813 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:52:00.565770 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4fd9da2-13c7-4b85-9c89-f10a28d43049-metrics-certs\") pod \"network-metrics-daemon-x982g\" (UID: \"d4fd9da2-13c7-4b85-9c89-f10a28d43049\") " pod="openshift-multus/network-metrics-daemon-x982g" Sep 11 11:52:00.568312 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:52:00.568286 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4fd9da2-13c7-4b85-9c89-f10a28d43049-metrics-certs\") pod \"network-metrics-daemon-x982g\" (UID: \"d4fd9da2-13c7-4b85-9c89-f10a28d43049\") " pod="openshift-multus/network-metrics-daemon-x982g" Sep 11 11:52:00.657292 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:52:00.657260 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5sht6\"" Sep 11 11:52:00.664929 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:52:00.664911 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x982g" Sep 11 11:52:00.939407 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:52:00.939255 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-x982g"] Sep 11 11:52:00.941749 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:52:00.941722 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4fd9da2_13c7_4b85_9c89_f10a28d43049.slice/crio-55dcc40be436da8485a48a9e993303e9f21430371af29c09bde0c0d89006ba7e WatchSource:0}: Error finding container 55dcc40be436da8485a48a9e993303e9f21430371af29c09bde0c0d89006ba7e: Status 404 returned error can't find the container with id 55dcc40be436da8485a48a9e993303e9f21430371af29c09bde0c0d89006ba7e Sep 11 11:52:01.619514 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:52:01.619475 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-x982g" event={"ID":"d4fd9da2-13c7-4b85-9c89-f10a28d43049","Type":"ContainerStarted","Data":"55dcc40be436da8485a48a9e993303e9f21430371af29c09bde0c0d89006ba7e"} Sep 11 11:52:02.624410 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:52:02.624373 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-x982g" event={"ID":"d4fd9da2-13c7-4b85-9c89-f10a28d43049","Type":"ContainerStarted","Data":"8670f8b4e0bc32005447c89e01d2c8f4f6403669b67f4bc61415ca368c0f3c1f"} Sep 11 11:52:02.624789 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:52:02.624431 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-x982g" event={"ID":"d4fd9da2-13c7-4b85-9c89-f10a28d43049","Type":"ContainerStarted","Data":"5f8ec23dd2c477208b0673c43ece05207bbb2655cc638b185699988d58740226"} Sep 11 11:52:02.643458 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:52:02.643411 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-x982g" podStartSLOduration=253.63282212 podStartE2EDuration="4m14.643396101s" totalImagesPullingTime="1.010573981s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:47:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:52:02.64201217 +0000 UTC m=+254.406985345" watchObservedRunningTime="2026-09-11 11:52:02.643396101 +0000 UTC m=+254.408369276" Sep 11 11:52:48.666019 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:52:48.665994 2566 kubelet.go:1648] "Image garbage collection succeeded" Sep 11 11:54:22.778219 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:22.778184 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-4tc4c"] Sep 11 11:54:22.778676 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:22.778414 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60eec8ae-8c59-40f6-820a-9671b9ec919c" containerName="registry" Sep 11 11:54:22.778676 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:22.778425 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="60eec8ae-8c59-40f6-820a-9671b9ec919c" containerName="registry" Sep 11 11:54:22.778676 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:22.778463 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="60eec8ae-8c59-40f6-820a-9671b9ec919c" containerName="registry" Sep 11 11:54:22.781038 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:22.781022 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-4tc4c" Sep 11 11:54:22.786274 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:22.786253 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Sep 11 11:54:22.786647 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:22.786629 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-58v49\"" Sep 11 11:54:22.786726 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:22.786676 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Sep 11 11:54:22.794653 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:22.794626 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-4tc4c"] Sep 11 11:54:22.835817 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:22.835787 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9550e554-8d1c-475b-892f-deddcd73722e-bound-sa-token\") pod \"cert-manager-55b4d954d5-4tc4c\" (UID: \"9550e554-8d1c-475b-892f-deddcd73722e\") " pod="cert-manager/cert-manager-55b4d954d5-4tc4c" Sep 11 11:54:22.835950 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:22.835834 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwft7\" (UniqueName: \"kubernetes.io/projected/9550e554-8d1c-475b-892f-deddcd73722e-kube-api-access-qwft7\") pod \"cert-manager-55b4d954d5-4tc4c\" (UID: \"9550e554-8d1c-475b-892f-deddcd73722e\") " pod="cert-manager/cert-manager-55b4d954d5-4tc4c" Sep 11 11:54:22.936409 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:22.936363 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9550e554-8d1c-475b-892f-deddcd73722e-bound-sa-token\") pod \"cert-manager-55b4d954d5-4tc4c\" (UID: \"9550e554-8d1c-475b-892f-deddcd73722e\") " pod="cert-manager/cert-manager-55b4d954d5-4tc4c" Sep 11 11:54:22.936578 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:22.936509 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwft7\" (UniqueName: \"kubernetes.io/projected/9550e554-8d1c-475b-892f-deddcd73722e-kube-api-access-qwft7\") pod \"cert-manager-55b4d954d5-4tc4c\" (UID: \"9550e554-8d1c-475b-892f-deddcd73722e\") " pod="cert-manager/cert-manager-55b4d954d5-4tc4c" Sep 11 11:54:22.945696 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:22.945666 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9550e554-8d1c-475b-892f-deddcd73722e-bound-sa-token\") pod \"cert-manager-55b4d954d5-4tc4c\" (UID: \"9550e554-8d1c-475b-892f-deddcd73722e\") " pod="cert-manager/cert-manager-55b4d954d5-4tc4c" Sep 11 11:54:22.945793 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:22.945727 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwft7\" (UniqueName: \"kubernetes.io/projected/9550e554-8d1c-475b-892f-deddcd73722e-kube-api-access-qwft7\") pod \"cert-manager-55b4d954d5-4tc4c\" (UID: \"9550e554-8d1c-475b-892f-deddcd73722e\") " pod="cert-manager/cert-manager-55b4d954d5-4tc4c" Sep 11 11:54:23.089630 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:23.089597 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-4tc4c" Sep 11 11:54:23.288148 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:23.288115 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r4pk2"] Sep 11 11:54:23.291472 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:23.291455 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r4pk2" Sep 11 11:54:23.295255 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:23.295228 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Sep 11 11:54:23.295504 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:23.295475 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Sep 11 11:54:23.298128 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:23.298107 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-wlcqq\"" Sep 11 11:54:23.304650 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:23.304628 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r4pk2"] Sep 11 11:54:23.339354 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:23.339326 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhmbc\" (UniqueName: \"kubernetes.io/projected/19402fa6-d6fd-456b-8e28-5a82dc39f678-kube-api-access-xhmbc\") pod \"openshift-lws-operator-fd8ccff4c-r4pk2\" (UID: \"19402fa6-d6fd-456b-8e28-5a82dc39f678\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r4pk2" Sep 11 11:54:23.339492 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:23.339371 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/19402fa6-d6fd-456b-8e28-5a82dc39f678-tmp\") pod \"openshift-lws-operator-fd8ccff4c-r4pk2\" (UID: \"19402fa6-d6fd-456b-8e28-5a82dc39f678\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r4pk2" Sep 11 11:54:23.362982 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:23.362914 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-4tc4c"] Sep 11 11:54:23.366267 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:54:23.366244 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9550e554_8d1c_475b_892f_deddcd73722e.slice/crio-00aeba1cee5f0023ed04d0957d18a737be63b205d9befcfae8e4bbc7c9d80ba2 WatchSource:0}: Error finding container 00aeba1cee5f0023ed04d0957d18a737be63b205d9befcfae8e4bbc7c9d80ba2: Status 404 returned error can't find the container with id 00aeba1cee5f0023ed04d0957d18a737be63b205d9befcfae8e4bbc7c9d80ba2 Sep 11 11:54:23.368153 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:23.368136 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 11 11:54:23.440077 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:23.440047 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/19402fa6-d6fd-456b-8e28-5a82dc39f678-tmp\") pod \"openshift-lws-operator-fd8ccff4c-r4pk2\" (UID: \"19402fa6-d6fd-456b-8e28-5a82dc39f678\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r4pk2" Sep 11 11:54:23.440242 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:23.440105 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhmbc\" (UniqueName: \"kubernetes.io/projected/19402fa6-d6fd-456b-8e28-5a82dc39f678-kube-api-access-xhmbc\") pod \"openshift-lws-operator-fd8ccff4c-r4pk2\" (UID: \"19402fa6-d6fd-456b-8e28-5a82dc39f678\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r4pk2" Sep 11 11:54:23.440501 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:23.440481 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/19402fa6-d6fd-456b-8e28-5a82dc39f678-tmp\") pod \"openshift-lws-operator-fd8ccff4c-r4pk2\" (UID: \"19402fa6-d6fd-456b-8e28-5a82dc39f678\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r4pk2" Sep 11 11:54:23.451837 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:23.451816 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhmbc\" (UniqueName: \"kubernetes.io/projected/19402fa6-d6fd-456b-8e28-5a82dc39f678-kube-api-access-xhmbc\") pod \"openshift-lws-operator-fd8ccff4c-r4pk2\" (UID: \"19402fa6-d6fd-456b-8e28-5a82dc39f678\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r4pk2" Sep 11 11:54:23.599696 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:23.599661 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r4pk2" Sep 11 11:54:23.871080 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:23.870837 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r4pk2"] Sep 11 11:54:23.874010 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:54:23.873968 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod19402fa6_d6fd_456b_8e28_5a82dc39f678.slice/crio-82bb1e6a2a556ec3bdefae4bea02f29a9bfe2379e071404942b3f8202599ad62 WatchSource:0}: Error finding container 82bb1e6a2a556ec3bdefae4bea02f29a9bfe2379e071404942b3f8202599ad62: Status 404 returned error can't find the container with id 82bb1e6a2a556ec3bdefae4bea02f29a9bfe2379e071404942b3f8202599ad62 Sep 11 11:54:24.075166 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:24.075124 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-4tc4c" event={"ID":"9550e554-8d1c-475b-892f-deddcd73722e","Type":"ContainerStarted","Data":"00aeba1cee5f0023ed04d0957d18a737be63b205d9befcfae8e4bbc7c9d80ba2"} Sep 11 11:54:24.077169 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:24.077136 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r4pk2" event={"ID":"19402fa6-d6fd-456b-8e28-5a82dc39f678","Type":"ContainerStarted","Data":"82bb1e6a2a556ec3bdefae4bea02f29a9bfe2379e071404942b3f8202599ad62"} Sep 11 11:54:28.100818 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:28.100781 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-4tc4c" event={"ID":"9550e554-8d1c-475b-892f-deddcd73722e","Type":"ContainerStarted","Data":"8976a862bb603861361fef06ede6c76bb00aff75d8d4b7aeab93ac83cbb1c25e"} Sep 11 11:54:28.102510 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:28.102482 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r4pk2" event={"ID":"19402fa6-d6fd-456b-8e28-5a82dc39f678","Type":"ContainerStarted","Data":"a5b63b58f61fc44c85bd1592d8e1750448f77a6463fb696419e36bd600ed627b"} Sep 11 11:54:28.118252 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:28.118212 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-4tc4c" podStartSLOduration=1.995255341 podStartE2EDuration="6.118199119s" totalImagesPullingTime="4.122943778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:54:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:54:28.116986139 +0000 UTC m=+399.881959314" watchObservedRunningTime="2026-09-11 11:54:28.118199119 +0000 UTC m=+399.883172294" Sep 11 11:54:28.135612 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:28.135567 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r4pk2" podStartSLOduration=1.470752614 podStartE2EDuration="5.135552863s" totalImagesPullingTime="3.664800249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:54:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:54:28.13470681 +0000 UTC m=+399.899679988" watchObservedRunningTime="2026-09-11 11:54:28.135552863 +0000 UTC m=+399.900526037" Sep 11 11:54:47.908516 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:47.908480 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7fb8846894-zcf9q"] Sep 11 11:54:47.911504 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:47.911487 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-zcf9q" Sep 11 11:54:47.916307 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:47.915983 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 11 11:54:47.919549 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:47.916525 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 11 11:54:47.919549 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:47.916801 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 11 11:54:47.919549 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:47.917402 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 11 11:54:47.919549 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:47.917511 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-47w2g\"" Sep 11 11:54:47.935612 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:47.935585 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7fb8846894-zcf9q"] Sep 11 11:54:48.016785 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:48.016749 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6bc4\" (UniqueName: \"kubernetes.io/projected/77edd868-8a77-4e81-8bf9-301419e19c8d-kube-api-access-j6bc4\") pod \"opendatahub-operator-controller-manager-7fb8846894-zcf9q\" (UID: \"77edd868-8a77-4e81-8bf9-301419e19c8d\") " pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-zcf9q" Sep 11 11:54:48.016956 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:48.016798 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/77edd868-8a77-4e81-8bf9-301419e19c8d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7fb8846894-zcf9q\" (UID: \"77edd868-8a77-4e81-8bf9-301419e19c8d\") " pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-zcf9q" Sep 11 11:54:48.016956 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:48.016825 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/77edd868-8a77-4e81-8bf9-301419e19c8d-manifests\") pod \"opendatahub-operator-controller-manager-7fb8846894-zcf9q\" (UID: \"77edd868-8a77-4e81-8bf9-301419e19c8d\") " pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-zcf9q" Sep 11 11:54:48.016956 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:48.016886 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77edd868-8a77-4e81-8bf9-301419e19c8d-tmp\") pod \"opendatahub-operator-controller-manager-7fb8846894-zcf9q\" (UID: \"77edd868-8a77-4e81-8bf9-301419e19c8d\") " pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-zcf9q" Sep 11 11:54:48.016956 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:48.016932 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/77edd868-8a77-4e81-8bf9-301419e19c8d-webhook-cert\") pod \"opendatahub-operator-controller-manager-7fb8846894-zcf9q\" (UID: \"77edd868-8a77-4e81-8bf9-301419e19c8d\") " pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-zcf9q" Sep 11 11:54:48.118249 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:48.118212 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/77edd868-8a77-4e81-8bf9-301419e19c8d-manifests\") pod \"opendatahub-operator-controller-manager-7fb8846894-zcf9q\" (UID: \"77edd868-8a77-4e81-8bf9-301419e19c8d\") " pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-zcf9q" Sep 11 11:54:48.118488 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:48.118279 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77edd868-8a77-4e81-8bf9-301419e19c8d-tmp\") pod \"opendatahub-operator-controller-manager-7fb8846894-zcf9q\" (UID: \"77edd868-8a77-4e81-8bf9-301419e19c8d\") " pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-zcf9q" Sep 11 11:54:48.118488 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:48.118313 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/77edd868-8a77-4e81-8bf9-301419e19c8d-webhook-cert\") pod \"opendatahub-operator-controller-manager-7fb8846894-zcf9q\" (UID: \"77edd868-8a77-4e81-8bf9-301419e19c8d\") " pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-zcf9q" Sep 11 11:54:48.118488 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:48.118343 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6bc4\" (UniqueName: \"kubernetes.io/projected/77edd868-8a77-4e81-8bf9-301419e19c8d-kube-api-access-j6bc4\") pod \"opendatahub-operator-controller-manager-7fb8846894-zcf9q\" (UID: \"77edd868-8a77-4e81-8bf9-301419e19c8d\") " pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-zcf9q" Sep 11 11:54:48.118488 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:48.118468 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/77edd868-8a77-4e81-8bf9-301419e19c8d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7fb8846894-zcf9q\" (UID: \"77edd868-8a77-4e81-8bf9-301419e19c8d\") " pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-zcf9q" Sep 11 11:54:48.118717 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:48.118526 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/77edd868-8a77-4e81-8bf9-301419e19c8d-manifests\") pod \"opendatahub-operator-controller-manager-7fb8846894-zcf9q\" (UID: \"77edd868-8a77-4e81-8bf9-301419e19c8d\") " pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-zcf9q" Sep 11 11:54:48.120449 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:48.120419 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77edd868-8a77-4e81-8bf9-301419e19c8d-tmp\") pod \"opendatahub-operator-controller-manager-7fb8846894-zcf9q\" (UID: \"77edd868-8a77-4e81-8bf9-301419e19c8d\") " pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-zcf9q" Sep 11 11:54:48.120748 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:48.120731 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/77edd868-8a77-4e81-8bf9-301419e19c8d-webhook-cert\") pod \"opendatahub-operator-controller-manager-7fb8846894-zcf9q\" (UID: \"77edd868-8a77-4e81-8bf9-301419e19c8d\") " pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-zcf9q" Sep 11 11:54:48.120800 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:48.120786 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/77edd868-8a77-4e81-8bf9-301419e19c8d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7fb8846894-zcf9q\" (UID: \"77edd868-8a77-4e81-8bf9-301419e19c8d\") " pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-zcf9q" Sep 11 11:54:48.136376 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:48.136350 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6bc4\" (UniqueName: \"kubernetes.io/projected/77edd868-8a77-4e81-8bf9-301419e19c8d-kube-api-access-j6bc4\") pod \"opendatahub-operator-controller-manager-7fb8846894-zcf9q\" (UID: \"77edd868-8a77-4e81-8bf9-301419e19c8d\") " pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-zcf9q" Sep 11 11:54:48.226105 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:48.226020 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-zcf9q" Sep 11 11:54:48.516654 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:48.516627 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7fb8846894-zcf9q"] Sep 11 11:54:48.518217 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:54:48.518185 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77edd868_8a77_4e81_8bf9_301419e19c8d.slice/crio-38b362686b539df9db72471734eaaee314049c1a87318aff4487a7543490cfaf WatchSource:0}: Error finding container 38b362686b539df9db72471734eaaee314049c1a87318aff4487a7543490cfaf: Status 404 returned error can't find the container with id 38b362686b539df9db72471734eaaee314049c1a87318aff4487a7543490cfaf Sep 11 11:54:49.173922 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:49.173889 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-zcf9q" event={"ID":"77edd868-8a77-4e81-8bf9-301419e19c8d","Type":"ContainerStarted","Data":"38b362686b539df9db72471734eaaee314049c1a87318aff4487a7543490cfaf"} Sep 11 11:54:52.189517 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:52.189475 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-zcf9q" event={"ID":"77edd868-8a77-4e81-8bf9-301419e19c8d","Type":"ContainerStarted","Data":"c6ffad6d152a9a5d2c069493bc0a8f9d475ae97e4669cb09ae3b02912da683d7"} Sep 11 11:54:53.195038 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:53.194995 2566 generic.go:356] "Generic (PLEG): container finished" podID="77edd868-8a77-4e81-8bf9-301419e19c8d" containerID="c6ffad6d152a9a5d2c069493bc0a8f9d475ae97e4669cb09ae3b02912da683d7" exitCode=0 Sep 11 11:54:53.195423 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:53.195059 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-zcf9q" event={"ID":"77edd868-8a77-4e81-8bf9-301419e19c8d","Type":"ContainerDied","Data":"c6ffad6d152a9a5d2c069493bc0a8f9d475ae97e4669cb09ae3b02912da683d7"} Sep 11 11:54:54.201231 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:54.201189 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-zcf9q" event={"ID":"77edd868-8a77-4e81-8bf9-301419e19c8d","Type":"ContainerStarted","Data":"9169a8d518b5226ac5b4fb2f170de01178a29779f283fc94b4d32bc01a76725f"} Sep 11 11:54:54.201815 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:54.201332 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-zcf9q" Sep 11 11:54:54.228653 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:54:54.228591 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-zcf9q" podStartSLOduration=2.901837715 podStartE2EDuration="7.228575942s" totalImagesPullingTime="3.323419133s" totalInitContainerRuntime="1.003319094s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:54:47 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:54:54.22548479 +0000 UTC m=+425.990457965" watchObservedRunningTime="2026-09-11 11:54:54.228575942 +0000 UTC m=+425.993549177" Sep 11 11:55:05.206425 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:05.206385 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-zcf9q" Sep 11 11:55:21.718540 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:21.718503 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5"] Sep 11 11:55:21.721601 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:21.721573 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5" Sep 11 11:55:21.725802 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:21.725779 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 11 11:55:21.726074 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:21.726049 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-kqfs5\"" Sep 11 11:55:21.726241 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:21.726219 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 11 11:55:21.726887 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:21.726855 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 11 11:55:21.737349 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:21.737323 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5"] Sep 11 11:55:21.770847 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:21.770814 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63b1165a-60a3-4edc-b78d-af87251b58ec-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5\" (UID: \"63b1165a-60a3-4edc-b78d-af87251b58ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5" Sep 11 11:55:21.771038 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:21.770860 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63b1165a-60a3-4edc-b78d-af87251b58ec-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5\" (UID: \"63b1165a-60a3-4edc-b78d-af87251b58ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5" Sep 11 11:55:21.771038 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:21.770897 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63b1165a-60a3-4edc-b78d-af87251b58ec-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5\" (UID: \"63b1165a-60a3-4edc-b78d-af87251b58ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5" Sep 11 11:55:21.771038 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:21.770939 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63b1165a-60a3-4edc-b78d-af87251b58ec-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5\" (UID: \"63b1165a-60a3-4edc-b78d-af87251b58ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5" Sep 11 11:55:21.771159 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:21.771033 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63b1165a-60a3-4edc-b78d-af87251b58ec-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5\" (UID: \"63b1165a-60a3-4edc-b78d-af87251b58ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5" Sep 11 11:55:21.771159 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:21.771083 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpqxm\" (UniqueName: \"kubernetes.io/projected/63b1165a-60a3-4edc-b78d-af87251b58ec-kube-api-access-zpqxm\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5\" (UID: \"63b1165a-60a3-4edc-b78d-af87251b58ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5" Sep 11 11:55:21.771159 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:21.771120 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63b1165a-60a3-4edc-b78d-af87251b58ec-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5\" (UID: \"63b1165a-60a3-4edc-b78d-af87251b58ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5" Sep 11 11:55:21.771263 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:21.771158 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63b1165a-60a3-4edc-b78d-af87251b58ec-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5\" (UID: \"63b1165a-60a3-4edc-b78d-af87251b58ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5" Sep 11 11:55:21.771263 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:21.771232 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63b1165a-60a3-4edc-b78d-af87251b58ec-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5\" (UID: \"63b1165a-60a3-4edc-b78d-af87251b58ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5" Sep 11 11:55:21.872510 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:21.872476 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63b1165a-60a3-4edc-b78d-af87251b58ec-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5\" (UID: \"63b1165a-60a3-4edc-b78d-af87251b58ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5" Sep 11 11:55:21.872700 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:21.872546 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63b1165a-60a3-4edc-b78d-af87251b58ec-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5\" (UID: \"63b1165a-60a3-4edc-b78d-af87251b58ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5" Sep 11 11:55:21.872700 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:21.872577 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63b1165a-60a3-4edc-b78d-af87251b58ec-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5\" (UID: \"63b1165a-60a3-4edc-b78d-af87251b58ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5" Sep 11 11:55:21.872700 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:21.872611 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63b1165a-60a3-4edc-b78d-af87251b58ec-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5\" (UID: \"63b1165a-60a3-4edc-b78d-af87251b58ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5" Sep 11 11:55:21.872850 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:21.872778 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63b1165a-60a3-4edc-b78d-af87251b58ec-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5\" (UID: \"63b1165a-60a3-4edc-b78d-af87251b58ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5" Sep 11 11:55:21.872850 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:21.872832 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpqxm\" (UniqueName: \"kubernetes.io/projected/63b1165a-60a3-4edc-b78d-af87251b58ec-kube-api-access-zpqxm\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5\" (UID: \"63b1165a-60a3-4edc-b78d-af87251b58ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5" Sep 11 11:55:21.873008 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:21.872897 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63b1165a-60a3-4edc-b78d-af87251b58ec-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5\" (UID: \"63b1165a-60a3-4edc-b78d-af87251b58ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5" Sep 11 11:55:21.873008 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:21.872933 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63b1165a-60a3-4edc-b78d-af87251b58ec-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5\" (UID: \"63b1165a-60a3-4edc-b78d-af87251b58ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5" Sep 11 11:55:21.873008 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:21.872966 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63b1165a-60a3-4edc-b78d-af87251b58ec-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5\" (UID: \"63b1165a-60a3-4edc-b78d-af87251b58ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5" Sep 11 11:55:21.873202 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:21.873177 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63b1165a-60a3-4edc-b78d-af87251b58ec-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5\" (UID: \"63b1165a-60a3-4edc-b78d-af87251b58ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5" Sep 11 11:55:21.873336 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:21.873205 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63b1165a-60a3-4edc-b78d-af87251b58ec-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5\" (UID: \"63b1165a-60a3-4edc-b78d-af87251b58ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5" Sep 11 11:55:21.873336 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:21.873276 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63b1165a-60a3-4edc-b78d-af87251b58ec-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5\" (UID: \"63b1165a-60a3-4edc-b78d-af87251b58ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5" Sep 11 11:55:21.873336 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:21.873332 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63b1165a-60a3-4edc-b78d-af87251b58ec-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5\" (UID: \"63b1165a-60a3-4edc-b78d-af87251b58ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5" Sep 11 11:55:21.873502 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:21.873384 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63b1165a-60a3-4edc-b78d-af87251b58ec-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5\" (UID: \"63b1165a-60a3-4edc-b78d-af87251b58ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5" Sep 11 11:55:21.874918 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:21.874897 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63b1165a-60a3-4edc-b78d-af87251b58ec-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5\" (UID: \"63b1165a-60a3-4edc-b78d-af87251b58ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5" Sep 11 11:55:21.875341 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:21.875317 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63b1165a-60a3-4edc-b78d-af87251b58ec-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5\" (UID: \"63b1165a-60a3-4edc-b78d-af87251b58ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5" Sep 11 11:55:21.881282 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:21.881264 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63b1165a-60a3-4edc-b78d-af87251b58ec-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5\" (UID: \"63b1165a-60a3-4edc-b78d-af87251b58ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5" Sep 11 11:55:21.881742 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:21.881721 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpqxm\" (UniqueName: \"kubernetes.io/projected/63b1165a-60a3-4edc-b78d-af87251b58ec-kube-api-access-zpqxm\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5\" (UID: \"63b1165a-60a3-4edc-b78d-af87251b58ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5" Sep 11 11:55:22.032297 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:22.032201 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5" Sep 11 11:55:22.311054 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:22.310917 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5"] Sep 11 11:55:22.313756 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:55:22.313723 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63b1165a_60a3_4edc_b78d_af87251b58ec.slice/crio-f30c4e8fbaa988fd46c537b7c89c6e254460d8f869a322579fe8b16ff37e43bc WatchSource:0}: Error finding container f30c4e8fbaa988fd46c537b7c89c6e254460d8f869a322579fe8b16ff37e43bc: Status 404 returned error can't find the container with id f30c4e8fbaa988fd46c537b7c89c6e254460d8f869a322579fe8b16ff37e43bc Sep 11 11:55:23.308265 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:23.308201 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5" event={"ID":"63b1165a-60a3-4edc-b78d-af87251b58ec","Type":"ContainerStarted","Data":"f30c4e8fbaa988fd46c537b7c89c6e254460d8f869a322579fe8b16ff37e43bc"} Sep 11 11:55:25.138728 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:25.138685 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 11:55:25.139020 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:25.138766 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 11:55:25.139020 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:25.138802 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 11:55:25.319154 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:25.319118 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5" event={"ID":"63b1165a-60a3-4edc-b78d-af87251b58ec","Type":"ContainerStarted","Data":"5a221c78e57ed3bcd5e63216156506a2f3a1f71cd549902a62e9fee86e59e291"} Sep 11 11:55:25.367511 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:25.367460 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5" podStartSLOduration=1.54767169 podStartE2EDuration="4.367444376s" totalImagesPullingTime="2.819772686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:55:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:55:25.365803681 +0000 UTC m=+457.130776866" watchObservedRunningTime="2026-09-11 11:55:25.367444376 +0000 UTC m=+457.132417552" Sep 11 11:55:26.033234 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:26.033196 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5" Sep 11 11:55:26.037788 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:26.037757 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5" Sep 11 11:55:26.322365 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:26.322336 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5" Sep 11 11:55:26.323307 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:26.323286 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5" Sep 11 11:55:26.352315 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:26.352278 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-866dd874d6-zmnzz"] Sep 11 11:55:26.355134 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:26.355115 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-866dd874d6-zmnzz" Sep 11 11:55:26.357708 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:26.357680 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Sep 11 11:55:26.358110 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:26.358088 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-2fddf\"" Sep 11 11:55:26.358200 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:26.358121 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Sep 11 11:55:26.358200 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:26.358127 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Sep 11 11:55:26.366528 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:26.366505 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-866dd874d6-zmnzz"] Sep 11 11:55:26.407036 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:26.407003 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldwbv\" (UniqueName: \"kubernetes.io/projected/b4943790-c1ba-4494-8f60-9b8a2160f5ab-kube-api-access-ldwbv\") pod \"lws-controller-manager-866dd874d6-zmnzz\" (UID: \"b4943790-c1ba-4494-8f60-9b8a2160f5ab\") " pod="openshift-lws-operator/lws-controller-manager-866dd874d6-zmnzz" Sep 11 11:55:26.407203 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:26.407077 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b4943790-c1ba-4494-8f60-9b8a2160f5ab-metrics-cert\") pod \"lws-controller-manager-866dd874d6-zmnzz\" (UID: \"b4943790-c1ba-4494-8f60-9b8a2160f5ab\") " pod="openshift-lws-operator/lws-controller-manager-866dd874d6-zmnzz" Sep 11 11:55:26.407203 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:26.407140 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b4943790-c1ba-4494-8f60-9b8a2160f5ab-manager-config\") pod \"lws-controller-manager-866dd874d6-zmnzz\" (UID: \"b4943790-c1ba-4494-8f60-9b8a2160f5ab\") " pod="openshift-lws-operator/lws-controller-manager-866dd874d6-zmnzz" Sep 11 11:55:26.407203 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:26.407183 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b4943790-c1ba-4494-8f60-9b8a2160f5ab-cert\") pod \"lws-controller-manager-866dd874d6-zmnzz\" (UID: \"b4943790-c1ba-4494-8f60-9b8a2160f5ab\") " pod="openshift-lws-operator/lws-controller-manager-866dd874d6-zmnzz" Sep 11 11:55:26.508451 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:26.508349 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldwbv\" (UniqueName: \"kubernetes.io/projected/b4943790-c1ba-4494-8f60-9b8a2160f5ab-kube-api-access-ldwbv\") pod \"lws-controller-manager-866dd874d6-zmnzz\" (UID: \"b4943790-c1ba-4494-8f60-9b8a2160f5ab\") " pod="openshift-lws-operator/lws-controller-manager-866dd874d6-zmnzz" Sep 11 11:55:26.508671 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:26.508511 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b4943790-c1ba-4494-8f60-9b8a2160f5ab-metrics-cert\") pod \"lws-controller-manager-866dd874d6-zmnzz\" (UID: \"b4943790-c1ba-4494-8f60-9b8a2160f5ab\") " pod="openshift-lws-operator/lws-controller-manager-866dd874d6-zmnzz" Sep 11 11:55:26.508671 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:26.508537 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b4943790-c1ba-4494-8f60-9b8a2160f5ab-manager-config\") pod \"lws-controller-manager-866dd874d6-zmnzz\" (UID: \"b4943790-c1ba-4494-8f60-9b8a2160f5ab\") " pod="openshift-lws-operator/lws-controller-manager-866dd874d6-zmnzz" Sep 11 11:55:26.508671 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:26.508556 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b4943790-c1ba-4494-8f60-9b8a2160f5ab-cert\") pod \"lws-controller-manager-866dd874d6-zmnzz\" (UID: \"b4943790-c1ba-4494-8f60-9b8a2160f5ab\") " pod="openshift-lws-operator/lws-controller-manager-866dd874d6-zmnzz" Sep 11 11:55:26.509213 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:26.509182 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b4943790-c1ba-4494-8f60-9b8a2160f5ab-manager-config\") pod \"lws-controller-manager-866dd874d6-zmnzz\" (UID: \"b4943790-c1ba-4494-8f60-9b8a2160f5ab\") " pod="openshift-lws-operator/lws-controller-manager-866dd874d6-zmnzz" Sep 11 11:55:26.510993 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:26.510969 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b4943790-c1ba-4494-8f60-9b8a2160f5ab-cert\") pod \"lws-controller-manager-866dd874d6-zmnzz\" (UID: \"b4943790-c1ba-4494-8f60-9b8a2160f5ab\") " pod="openshift-lws-operator/lws-controller-manager-866dd874d6-zmnzz" Sep 11 11:55:26.511100 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:26.511021 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b4943790-c1ba-4494-8f60-9b8a2160f5ab-metrics-cert\") pod \"lws-controller-manager-866dd874d6-zmnzz\" (UID: \"b4943790-c1ba-4494-8f60-9b8a2160f5ab\") " pod="openshift-lws-operator/lws-controller-manager-866dd874d6-zmnzz" Sep 11 11:55:26.523936 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:26.523911 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldwbv\" (UniqueName: \"kubernetes.io/projected/b4943790-c1ba-4494-8f60-9b8a2160f5ab-kube-api-access-ldwbv\") pod \"lws-controller-manager-866dd874d6-zmnzz\" (UID: \"b4943790-c1ba-4494-8f60-9b8a2160f5ab\") " pod="openshift-lws-operator/lws-controller-manager-866dd874d6-zmnzz" Sep 11 11:55:26.664984 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:26.664894 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-866dd874d6-zmnzz" Sep 11 11:55:26.955238 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:26.955184 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-866dd874d6-zmnzz"] Sep 11 11:55:26.957568 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:55:26.957532 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4943790_c1ba_4494_8f60_9b8a2160f5ab.slice/crio-5097318bd4d0cf57a351a6c7d54c8e6786fa393d9eeda617c20401b99275de2b WatchSource:0}: Error finding container 5097318bd4d0cf57a351a6c7d54c8e6786fa393d9eeda617c20401b99275de2b: Status 404 returned error can't find the container with id 5097318bd4d0cf57a351a6c7d54c8e6786fa393d9eeda617c20401b99275de2b Sep 11 11:55:27.327668 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:27.327632 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-866dd874d6-zmnzz" event={"ID":"b4943790-c1ba-4494-8f60-9b8a2160f5ab","Type":"ContainerStarted","Data":"5097318bd4d0cf57a351a6c7d54c8e6786fa393d9eeda617c20401b99275de2b"} Sep 11 11:55:29.337936 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:29.337895 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-866dd874d6-zmnzz" event={"ID":"b4943790-c1ba-4494-8f60-9b8a2160f5ab","Type":"ContainerStarted","Data":"6d20321aedb60ed61cc9221c773e35262073981315de0a38bf0a277c2ca1f416"} Sep 11 11:55:29.338369 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:29.338029 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-866dd874d6-zmnzz" Sep 11 11:55:29.372518 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:29.372476 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-866dd874d6-zmnzz" podStartSLOduration=1.865608141 podStartE2EDuration="3.37246121s" totalImagesPullingTime="1.506853069s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:55:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:55:29.370650465 +0000 UTC m=+461.135623797" watchObservedRunningTime="2026-09-11 11:55:29.37246121 +0000 UTC m=+461.137434386" Sep 11 11:55:40.344241 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:55:40.344206 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-866dd874d6-zmnzz" Sep 11 11:56:44.306345 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:44.306263 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nbtxt"] Sep 11 11:56:44.309289 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:44.309273 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nbtxt" Sep 11 11:56:44.314117 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:44.314093 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 11 11:56:44.314258 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:44.314138 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 11 11:56:44.314701 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:44.314686 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-7jt4t\"" Sep 11 11:56:44.336919 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:44.336890 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nbtxt"] Sep 11 11:56:44.503408 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:44.503361 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtdq5\" (UniqueName: \"kubernetes.io/projected/41b96966-b1f6-4510-b36c-0f6b0c3f9833-kube-api-access-vtdq5\") pod \"kuadrant-operator-controller-manager-76f8f686bf-nbtxt\" (UID: \"41b96966-b1f6-4510-b36c-0f6b0c3f9833\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nbtxt" Sep 11 11:56:44.503593 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:44.503417 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/41b96966-b1f6-4510-b36c-0f6b0c3f9833-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-nbtxt\" (UID: \"41b96966-b1f6-4510-b36c-0f6b0c3f9833\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nbtxt" Sep 11 11:56:44.604025 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:44.603995 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/41b96966-b1f6-4510-b36c-0f6b0c3f9833-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-nbtxt\" (UID: \"41b96966-b1f6-4510-b36c-0f6b0c3f9833\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nbtxt" Sep 11 11:56:44.604180 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:44.604058 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtdq5\" (UniqueName: \"kubernetes.io/projected/41b96966-b1f6-4510-b36c-0f6b0c3f9833-kube-api-access-vtdq5\") pod \"kuadrant-operator-controller-manager-76f8f686bf-nbtxt\" (UID: \"41b96966-b1f6-4510-b36c-0f6b0c3f9833\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nbtxt" Sep 11 11:56:44.604434 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:44.604408 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/41b96966-b1f6-4510-b36c-0f6b0c3f9833-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-nbtxt\" (UID: \"41b96966-b1f6-4510-b36c-0f6b0c3f9833\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nbtxt" Sep 11 11:56:44.616149 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:44.616120 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtdq5\" (UniqueName: \"kubernetes.io/projected/41b96966-b1f6-4510-b36c-0f6b0c3f9833-kube-api-access-vtdq5\") pod \"kuadrant-operator-controller-manager-76f8f686bf-nbtxt\" (UID: \"41b96966-b1f6-4510-b36c-0f6b0c3f9833\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nbtxt" Sep 11 11:56:44.618388 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:44.618365 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nbtxt" Sep 11 11:56:44.906303 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:44.906273 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nbtxt"] Sep 11 11:56:44.908227 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:56:44.908203 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41b96966_b1f6_4510_b36c_0f6b0c3f9833.slice/crio-52d3163a731edf55d58d65142c167e378557225c372ab81869b7db77744aa017 WatchSource:0}: Error finding container 52d3163a731edf55d58d65142c167e378557225c372ab81869b7db77744aa017: Status 404 returned error can't find the container with id 52d3163a731edf55d58d65142c167e378557225c372ab81869b7db77744aa017 Sep 11 11:56:45.626352 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:45.626315 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nbtxt" event={"ID":"41b96966-b1f6-4510-b36c-0f6b0c3f9833","Type":"ContainerStarted","Data":"52d3163a731edf55d58d65142c167e378557225c372ab81869b7db77744aa017"} Sep 11 11:56:46.335046 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:46.335009 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-gmx4v"] Sep 11 11:56:46.338460 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:46.338435 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-gmx4v" Sep 11 11:56:46.341550 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:46.341522 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-wzvlq\"" Sep 11 11:56:46.341786 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:46.341761 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Sep 11 11:56:46.361477 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:46.361411 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-gmx4v"] Sep 11 11:56:46.416285 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:46.416245 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slclv\" (UniqueName: \"kubernetes.io/projected/413baf89-6e9c-4338-9ee3-b09da8291801-kube-api-access-slclv\") pod \"dns-operator-controller-manager-7b84cd7c9c-gmx4v\" (UID: \"413baf89-6e9c-4338-9ee3-b09da8291801\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-gmx4v" Sep 11 11:56:46.516749 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:46.516685 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slclv\" (UniqueName: \"kubernetes.io/projected/413baf89-6e9c-4338-9ee3-b09da8291801-kube-api-access-slclv\") pod \"dns-operator-controller-manager-7b84cd7c9c-gmx4v\" (UID: \"413baf89-6e9c-4338-9ee3-b09da8291801\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-gmx4v" Sep 11 11:56:46.537149 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:46.537084 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slclv\" (UniqueName: \"kubernetes.io/projected/413baf89-6e9c-4338-9ee3-b09da8291801-kube-api-access-slclv\") pod \"dns-operator-controller-manager-7b84cd7c9c-gmx4v\" (UID: \"413baf89-6e9c-4338-9ee3-b09da8291801\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-gmx4v" Sep 11 11:56:46.653046 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:46.652853 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-gmx4v" Sep 11 11:56:46.952307 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:46.952194 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-gmx4v"] Sep 11 11:56:47.617846 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:56:47.617806 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod413baf89_6e9c_4338_9ee3_b09da8291801.slice/crio-59cd74a7ddd9b396315993307abf4cf9148cd0b8f2fe07c5af47d0a5623aebb8 WatchSource:0}: Error finding container 59cd74a7ddd9b396315993307abf4cf9148cd0b8f2fe07c5af47d0a5623aebb8: Status 404 returned error can't find the container with id 59cd74a7ddd9b396315993307abf4cf9148cd0b8f2fe07c5af47d0a5623aebb8 Sep 11 11:56:47.638731 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:47.638697 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-gmx4v" event={"ID":"413baf89-6e9c-4338-9ee3-b09da8291801","Type":"ContainerStarted","Data":"59cd74a7ddd9b396315993307abf4cf9148cd0b8f2fe07c5af47d0a5623aebb8"} Sep 11 11:56:49.651883 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:49.651828 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nbtxt" event={"ID":"41b96966-b1f6-4510-b36c-0f6b0c3f9833","Type":"ContainerStarted","Data":"df3727ae97b01104d6afabc71fcb376e7ea784918cbe14dc518dde1c29539062"} Sep 11 11:56:49.652338 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:49.651939 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nbtxt" Sep 11 11:56:49.680576 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:49.680416 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nbtxt" podStartSLOduration=1.3437006120000001 podStartE2EDuration="5.680395667s" totalImagesPullingTime="4.336695055s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:56:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:56:49.68036042 +0000 UTC m=+541.445333597" watchObservedRunningTime="2026-09-11 11:56:49.680395667 +0000 UTC m=+541.445368843" Sep 11 11:56:50.414460 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:50.414436 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Sep 11 11:56:50.657649 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:50.657607 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-gmx4v" event={"ID":"413baf89-6e9c-4338-9ee3-b09da8291801","Type":"ContainerStarted","Data":"046d7e468d2ff7ebf1b254407ec27d633c2bc7e063edb717f97643cf02da50be"} Sep 11 11:56:50.658212 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:50.657859 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-gmx4v" Sep 11 11:56:50.683796 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:50.683697 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-gmx4v" podStartSLOduration=1.8923053520000002 podStartE2EDuration="4.683682149s" totalImagesPullingTime="2.791376797s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:56:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:56:50.682480166 +0000 UTC m=+542.447453340" watchObservedRunningTime="2026-09-11 11:56:50.683682149 +0000 UTC m=+542.448655324" Sep 11 11:56:52.041651 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:52.041619 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-8dbr8"] Sep 11 11:56:52.044779 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:52.044756 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8dbr8" Sep 11 11:56:52.048007 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:52.047984 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Sep 11 11:56:52.048098 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:52.047990 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-h7kcd\"" Sep 11 11:56:52.048098 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:52.047990 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Sep 11 11:56:52.053655 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:52.053630 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-8dbr8"] Sep 11 11:56:52.056626 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:52.056603 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/835ff850-43f8-495f-813f-4d2950a8749e-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-8dbr8\" (UID: \"835ff850-43f8-495f-813f-4d2950a8749e\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8dbr8" Sep 11 11:56:52.056730 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:52.056668 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vn652\" (UniqueName: \"kubernetes.io/projected/835ff850-43f8-495f-813f-4d2950a8749e-kube-api-access-vn652\") pod \"kuadrant-console-plugin-fbdd4f445-8dbr8\" (UID: \"835ff850-43f8-495f-813f-4d2950a8749e\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8dbr8" Sep 11 11:56:52.056780 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:52.056724 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/835ff850-43f8-495f-813f-4d2950a8749e-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-8dbr8\" (UID: \"835ff850-43f8-495f-813f-4d2950a8749e\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8dbr8" Sep 11 11:56:52.157319 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:52.157285 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/835ff850-43f8-495f-813f-4d2950a8749e-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-8dbr8\" (UID: \"835ff850-43f8-495f-813f-4d2950a8749e\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8dbr8" Sep 11 11:56:52.157496 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:52.157338 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/835ff850-43f8-495f-813f-4d2950a8749e-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-8dbr8\" (UID: \"835ff850-43f8-495f-813f-4d2950a8749e\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8dbr8" Sep 11 11:56:52.157496 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:52.157405 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vn652\" (UniqueName: \"kubernetes.io/projected/835ff850-43f8-495f-813f-4d2950a8749e-kube-api-access-vn652\") pod \"kuadrant-console-plugin-fbdd4f445-8dbr8\" (UID: \"835ff850-43f8-495f-813f-4d2950a8749e\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8dbr8" Sep 11 11:56:52.158080 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:52.158061 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/835ff850-43f8-495f-813f-4d2950a8749e-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-8dbr8\" (UID: \"835ff850-43f8-495f-813f-4d2950a8749e\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8dbr8" Sep 11 11:56:52.159724 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:52.159696 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/835ff850-43f8-495f-813f-4d2950a8749e-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-8dbr8\" (UID: \"835ff850-43f8-495f-813f-4d2950a8749e\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8dbr8" Sep 11 11:56:52.166126 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:52.166100 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vn652\" (UniqueName: \"kubernetes.io/projected/835ff850-43f8-495f-813f-4d2950a8749e-kube-api-access-vn652\") pod \"kuadrant-console-plugin-fbdd4f445-8dbr8\" (UID: \"835ff850-43f8-495f-813f-4d2950a8749e\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8dbr8" Sep 11 11:56:52.355278 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:52.355238 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8dbr8" Sep 11 11:56:52.624667 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:52.624629 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-8dbr8"] Sep 11 11:56:52.627187 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:56:52.627152 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod835ff850_43f8_495f_813f_4d2950a8749e.slice/crio-5c59a51da2e1998893ac57b37afb642305838f3892776461d1ab6306242f9a8d WatchSource:0}: Error finding container 5c59a51da2e1998893ac57b37afb642305838f3892776461d1ab6306242f9a8d: Status 404 returned error can't find the container with id 5c59a51da2e1998893ac57b37afb642305838f3892776461d1ab6306242f9a8d Sep 11 11:56:52.666807 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:52.666777 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8dbr8" event={"ID":"835ff850-43f8-495f-813f-4d2950a8749e","Type":"ContainerStarted","Data":"5c59a51da2e1998893ac57b37afb642305838f3892776461d1ab6306242f9a8d"} Sep 11 11:56:57.695756 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:57.695717 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8dbr8" event={"ID":"835ff850-43f8-495f-813f-4d2950a8749e","Type":"ContainerStarted","Data":"a12a9786b57054d38b3cf39a2ddd3151fbbec994f0c7993d057d2085d85e47be"} Sep 11 11:56:57.711737 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:56:57.711678 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8dbr8" podStartSLOduration=1.33882607 podStartE2EDuration="5.711607507s" totalImagesPullingTime="4.372781437s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:56:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:56:57.71114029 +0000 UTC m=+549.476113490" watchObservedRunningTime="2026-09-11 11:56:57.711607507 +0000 UTC m=+549.476580684" Sep 11 11:57:00.660629 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:00.660595 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nbtxt" Sep 11 11:57:01.663281 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:01.663245 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-gmx4v" Sep 11 11:57:03.267279 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:03.267238 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nbtxt"] Sep 11 11:57:03.267719 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:03.267479 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nbtxt" podUID="41b96966-b1f6-4510-b36c-0f6b0c3f9833" containerName="manager" containerID="cri-o://df3727ae97b01104d6afabc71fcb376e7ea784918cbe14dc518dde1c29539062" gracePeriod=2 Sep 11 11:57:03.275708 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:03.275291 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nbtxt"] Sep 11 11:57:03.291921 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:03.291892 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4r2b9"] Sep 11 11:57:03.292164 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:03.292153 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41b96966-b1f6-4510-b36c-0f6b0c3f9833" containerName="manager" Sep 11 11:57:03.292220 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:03.292166 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="41b96966-b1f6-4510-b36c-0f6b0c3f9833" containerName="manager" Sep 11 11:57:03.292258 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:03.292221 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="41b96966-b1f6-4510-b36c-0f6b0c3f9833" containerName="manager" Sep 11 11:57:03.295109 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:03.295088 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4r2b9" Sep 11 11:57:03.301922 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:57:03.301890 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-nbtxt\" is forbidden: User \"system:node:ip-10-0-132-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-170.ec2.internal' and this object" podUID="41b96966-b1f6-4510-b36c-0f6b0c3f9833" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nbtxt" Sep 11 11:57:03.332117 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:03.332084 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4r2b9"] Sep 11 11:57:03.350952 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:03.350921 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/15a1dd89-5959-4508-aae7-254d5852b8fc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-4r2b9\" (UID: \"15a1dd89-5959-4508-aae7-254d5852b8fc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4r2b9" Sep 11 11:57:03.351157 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:03.350975 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxcvs\" (UniqueName: \"kubernetes.io/projected/15a1dd89-5959-4508-aae7-254d5852b8fc-kube-api-access-hxcvs\") pod \"kuadrant-operator-controller-manager-76f8f686bf-4r2b9\" (UID: \"15a1dd89-5959-4508-aae7-254d5852b8fc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4r2b9" Sep 11 11:57:03.452467 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:03.452435 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/15a1dd89-5959-4508-aae7-254d5852b8fc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-4r2b9\" (UID: \"15a1dd89-5959-4508-aae7-254d5852b8fc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4r2b9" Sep 11 11:57:03.452603 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:03.452496 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxcvs\" (UniqueName: \"kubernetes.io/projected/15a1dd89-5959-4508-aae7-254d5852b8fc-kube-api-access-hxcvs\") pod \"kuadrant-operator-controller-manager-76f8f686bf-4r2b9\" (UID: \"15a1dd89-5959-4508-aae7-254d5852b8fc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4r2b9" Sep 11 11:57:03.452924 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:03.452901 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/15a1dd89-5959-4508-aae7-254d5852b8fc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-4r2b9\" (UID: \"15a1dd89-5959-4508-aae7-254d5852b8fc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4r2b9" Sep 11 11:57:03.465978 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:03.465949 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxcvs\" (UniqueName: \"kubernetes.io/projected/15a1dd89-5959-4508-aae7-254d5852b8fc-kube-api-access-hxcvs\") pod \"kuadrant-operator-controller-manager-76f8f686bf-4r2b9\" (UID: \"15a1dd89-5959-4508-aae7-254d5852b8fc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4r2b9" Sep 11 11:57:03.511463 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:03.511434 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nbtxt" Sep 11 11:57:03.514679 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:57:03.514653 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-nbtxt\" is forbidden: User \"system:node:ip-10-0-132-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-170.ec2.internal' and this object" podUID="41b96966-b1f6-4510-b36c-0f6b0c3f9833" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nbtxt" Sep 11 11:57:03.553171 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:03.553084 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vtdq5\" (UniqueName: \"kubernetes.io/projected/41b96966-b1f6-4510-b36c-0f6b0c3f9833-kube-api-access-vtdq5\") pod \"41b96966-b1f6-4510-b36c-0f6b0c3f9833\" (UID: \"41b96966-b1f6-4510-b36c-0f6b0c3f9833\") " Sep 11 11:57:03.553171 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:03.553121 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/41b96966-b1f6-4510-b36c-0f6b0c3f9833-extensions-socket-volume\") pod \"41b96966-b1f6-4510-b36c-0f6b0c3f9833\" (UID: \"41b96966-b1f6-4510-b36c-0f6b0c3f9833\") " Sep 11 11:57:03.553667 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:03.553642 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41b96966-b1f6-4510-b36c-0f6b0c3f9833-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "41b96966-b1f6-4510-b36c-0f6b0c3f9833" (UID: "41b96966-b1f6-4510-b36c-0f6b0c3f9833"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 11:57:03.555303 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:03.555282 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41b96966-b1f6-4510-b36c-0f6b0c3f9833-kube-api-access-vtdq5" (OuterVolumeSpecName: "kube-api-access-vtdq5") pod "41b96966-b1f6-4510-b36c-0f6b0c3f9833" (UID: "41b96966-b1f6-4510-b36c-0f6b0c3f9833"). InnerVolumeSpecName "kube-api-access-vtdq5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 11:57:03.642085 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:03.642042 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4r2b9" Sep 11 11:57:03.654211 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:03.654186 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vtdq5\" (UniqueName: \"kubernetes.io/projected/41b96966-b1f6-4510-b36c-0f6b0c3f9833-kube-api-access-vtdq5\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 11:57:03.654298 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:03.654214 2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/41b96966-b1f6-4510-b36c-0f6b0c3f9833-extensions-socket-volume\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 11:57:03.722471 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:03.722437 2566 generic.go:356] "Generic (PLEG): container finished" podID="41b96966-b1f6-4510-b36c-0f6b0c3f9833" containerID="df3727ae97b01104d6afabc71fcb376e7ea784918cbe14dc518dde1c29539062" exitCode=0 Sep 11 11:57:03.722631 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:03.722476 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nbtxt" Sep 11 11:57:03.722631 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:03.722496 2566 scope.go:117] "RemoveContainer" containerID="df3727ae97b01104d6afabc71fcb376e7ea784918cbe14dc518dde1c29539062" Sep 11 11:57:03.727986 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:57:03.727950 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-nbtxt\" is forbidden: User \"system:node:ip-10-0-132-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-170.ec2.internal' and this object" podUID="41b96966-b1f6-4510-b36c-0f6b0c3f9833" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nbtxt" Sep 11 11:57:03.730530 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:03.730514 2566 scope.go:117] "RemoveContainer" containerID="df3727ae97b01104d6afabc71fcb376e7ea784918cbe14dc518dde1c29539062" Sep 11 11:57:03.730781 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:57:03.730763 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df3727ae97b01104d6afabc71fcb376e7ea784918cbe14dc518dde1c29539062\": container with ID starting with df3727ae97b01104d6afabc71fcb376e7ea784918cbe14dc518dde1c29539062 not found: ID does not exist" containerID="df3727ae97b01104d6afabc71fcb376e7ea784918cbe14dc518dde1c29539062" Sep 11 11:57:03.730821 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:03.730792 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df3727ae97b01104d6afabc71fcb376e7ea784918cbe14dc518dde1c29539062"} err="failed to get container status \"df3727ae97b01104d6afabc71fcb376e7ea784918cbe14dc518dde1c29539062\": rpc error: code = NotFound desc = could not find container \"df3727ae97b01104d6afabc71fcb376e7ea784918cbe14dc518dde1c29539062\": container with ID starting with df3727ae97b01104d6afabc71fcb376e7ea784918cbe14dc518dde1c29539062 not found: ID does not exist" Sep 11 11:57:03.732523 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:57:03.732500 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-nbtxt\" is forbidden: User \"system:node:ip-10-0-132-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-170.ec2.internal' and this object" podUID="41b96966-b1f6-4510-b36c-0f6b0c3f9833" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nbtxt" Sep 11 11:57:03.926124 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:03.926098 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4r2b9"] Sep 11 11:57:03.927802 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:57:03.927769 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15a1dd89_5959_4508_aae7_254d5852b8fc.slice/crio-2a0e0cd323adec400ddcac948498e217e8d0339768a75b19577e796d543b26e4 WatchSource:0}: Error finding container 2a0e0cd323adec400ddcac948498e217e8d0339768a75b19577e796d543b26e4: Status 404 returned error can't find the container with id 2a0e0cd323adec400ddcac948498e217e8d0339768a75b19577e796d543b26e4 Sep 11 11:57:04.435437 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:04.435399 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f292q"] Sep 11 11:57:04.438657 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:04.438639 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f292q" Sep 11 11:57:04.442522 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:57:04.442491 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-nbtxt\" is forbidden: User \"system:node:ip-10-0-132-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-170.ec2.internal' and this object" podUID="41b96966-b1f6-4510-b36c-0f6b0c3f9833" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nbtxt" Sep 11 11:57:04.461975 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:04.461941 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f292q"] Sep 11 11:57:04.561200 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:04.561160 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnz9w\" (UniqueName: \"kubernetes.io/projected/16e95dd7-18ae-4528-ab96-79201cf6079c-kube-api-access-pnz9w\") pod \"kuadrant-operator-controller-manager-5769f99d9f-f292q\" (UID: \"16e95dd7-18ae-4528-ab96-79201cf6079c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f292q" Sep 11 11:57:04.561397 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:04.561216 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/16e95dd7-18ae-4528-ab96-79201cf6079c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-f292q\" (UID: \"16e95dd7-18ae-4528-ab96-79201cf6079c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f292q" Sep 11 11:57:04.662348 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:04.662313 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnz9w\" (UniqueName: \"kubernetes.io/projected/16e95dd7-18ae-4528-ab96-79201cf6079c-kube-api-access-pnz9w\") pod \"kuadrant-operator-controller-manager-5769f99d9f-f292q\" (UID: \"16e95dd7-18ae-4528-ab96-79201cf6079c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f292q" Sep 11 11:57:04.662348 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:04.662354 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/16e95dd7-18ae-4528-ab96-79201cf6079c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-f292q\" (UID: \"16e95dd7-18ae-4528-ab96-79201cf6079c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f292q" Sep 11 11:57:04.662758 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:04.662742 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/16e95dd7-18ae-4528-ab96-79201cf6079c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-f292q\" (UID: \"16e95dd7-18ae-4528-ab96-79201cf6079c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f292q" Sep 11 11:57:04.674629 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:04.674595 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnz9w\" (UniqueName: \"kubernetes.io/projected/16e95dd7-18ae-4528-ab96-79201cf6079c-kube-api-access-pnz9w\") pod \"kuadrant-operator-controller-manager-5769f99d9f-f292q\" (UID: \"16e95dd7-18ae-4528-ab96-79201cf6079c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f292q" Sep 11 11:57:04.729499 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:04.729405 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4r2b9" event={"ID":"15a1dd89-5959-4508-aae7-254d5852b8fc","Type":"ContainerStarted","Data":"777cf37702881c8de5d3c35b36577528645dcefdae23876e46b0835ef1a7f9c8"} Sep 11 11:57:04.729499 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:04.729461 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4r2b9" event={"ID":"15a1dd89-5959-4508-aae7-254d5852b8fc","Type":"ContainerStarted","Data":"2a0e0cd323adec400ddcac948498e217e8d0339768a75b19577e796d543b26e4"} Sep 11 11:57:04.729729 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:04.729548 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4r2b9" Sep 11 11:57:04.735415 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:57:04.735379 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-nbtxt\" is forbidden: User \"system:node:ip-10-0-132-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-170.ec2.internal' and this object" podUID="41b96966-b1f6-4510-b36c-0f6b0c3f9833" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nbtxt" Sep 11 11:57:04.748088 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:04.748059 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f292q" Sep 11 11:57:04.757806 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:04.757773 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41b96966-b1f6-4510-b36c-0f6b0c3f9833" path="/var/lib/kubelet/pods/41b96966-b1f6-4510-b36c-0f6b0c3f9833/volumes" Sep 11 11:57:04.768368 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:04.768321 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4r2b9" podStartSLOduration=1.768303826 podStartE2EDuration="1.768303826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:57:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:57:04.767726081 +0000 UTC m=+556.532699291" watchObservedRunningTime="2026-09-11 11:57:04.768303826 +0000 UTC m=+556.533277002" Sep 11 11:57:05.031846 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:05.031818 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f292q"] Sep 11 11:57:05.033542 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:57:05.033514 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16e95dd7_18ae_4528_ab96_79201cf6079c.slice/crio-c216b22b26eb45b57f93ca8e884a83509792331eaa3a94104f573e8255e22c8e WatchSource:0}: Error finding container c216b22b26eb45b57f93ca8e884a83509792331eaa3a94104f573e8255e22c8e: Status 404 returned error can't find the container with id c216b22b26eb45b57f93ca8e884a83509792331eaa3a94104f573e8255e22c8e Sep 11 11:57:05.735521 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:05.735483 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f292q" event={"ID":"16e95dd7-18ae-4528-ab96-79201cf6079c","Type":"ContainerStarted","Data":"1d39d9d15da2b1855fb6dfae71ae9d6c605e912e35b8b86bcbef9833a084eb1c"} Sep 11 11:57:05.735986 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:05.735618 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f292q" Sep 11 11:57:05.735986 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:05.735637 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f292q" event={"ID":"16e95dd7-18ae-4528-ab96-79201cf6079c","Type":"ContainerStarted","Data":"c216b22b26eb45b57f93ca8e884a83509792331eaa3a94104f573e8255e22c8e"} Sep 11 11:57:05.760261 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:05.760209 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f292q" podStartSLOduration=1.760195397 podStartE2EDuration="1.760195397s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:57:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:57:05.758648454 +0000 UTC m=+557.523621628" watchObservedRunningTime="2026-09-11 11:57:05.760195397 +0000 UTC m=+557.525168572" Sep 11 11:57:15.737573 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:15.737528 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4r2b9" Sep 11 11:57:16.741827 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:16.741796 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f292q" Sep 11 11:57:16.804627 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:16.804590 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4r2b9"] Sep 11 11:57:16.804930 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:16.804898 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4r2b9" podUID="15a1dd89-5959-4508-aae7-254d5852b8fc" containerName="manager" containerID="cri-o://777cf37702881c8de5d3c35b36577528645dcefdae23876e46b0835ef1a7f9c8" gracePeriod=10 Sep 11 11:57:17.047917 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:17.047890 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4r2b9" Sep 11 11:57:17.152937 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:17.152902 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/15a1dd89-5959-4508-aae7-254d5852b8fc-extensions-socket-volume\") pod \"15a1dd89-5959-4508-aae7-254d5852b8fc\" (UID: \"15a1dd89-5959-4508-aae7-254d5852b8fc\") " Sep 11 11:57:17.153136 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:17.153016 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hxcvs\" (UniqueName: \"kubernetes.io/projected/15a1dd89-5959-4508-aae7-254d5852b8fc-kube-api-access-hxcvs\") pod \"15a1dd89-5959-4508-aae7-254d5852b8fc\" (UID: \"15a1dd89-5959-4508-aae7-254d5852b8fc\") " Sep 11 11:57:17.153305 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:17.153283 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15a1dd89-5959-4508-aae7-254d5852b8fc-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "15a1dd89-5959-4508-aae7-254d5852b8fc" (UID: "15a1dd89-5959-4508-aae7-254d5852b8fc"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 11:57:17.154942 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:17.154911 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15a1dd89-5959-4508-aae7-254d5852b8fc-kube-api-access-hxcvs" (OuterVolumeSpecName: "kube-api-access-hxcvs") pod "15a1dd89-5959-4508-aae7-254d5852b8fc" (UID: "15a1dd89-5959-4508-aae7-254d5852b8fc"). InnerVolumeSpecName "kube-api-access-hxcvs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 11:57:17.253920 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:17.253860 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxcvs\" (UniqueName: \"kubernetes.io/projected/15a1dd89-5959-4508-aae7-254d5852b8fc-kube-api-access-hxcvs\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 11:57:17.253920 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:17.253917 2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/15a1dd89-5959-4508-aae7-254d5852b8fc-extensions-socket-volume\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 11:57:17.790921 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:17.790884 2566 generic.go:356] "Generic (PLEG): container finished" podID="15a1dd89-5959-4508-aae7-254d5852b8fc" containerID="777cf37702881c8de5d3c35b36577528645dcefdae23876e46b0835ef1a7f9c8" exitCode=0 Sep 11 11:57:17.790921 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:17.790918 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4r2b9" Sep 11 11:57:17.791384 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:17.790940 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4r2b9" event={"ID":"15a1dd89-5959-4508-aae7-254d5852b8fc","Type":"ContainerDied","Data":"777cf37702881c8de5d3c35b36577528645dcefdae23876e46b0835ef1a7f9c8"} Sep 11 11:57:17.791384 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:17.790979 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4r2b9" event={"ID":"15a1dd89-5959-4508-aae7-254d5852b8fc","Type":"ContainerDied","Data":"2a0e0cd323adec400ddcac948498e217e8d0339768a75b19577e796d543b26e4"} Sep 11 11:57:17.791384 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:17.791000 2566 scope.go:117] "RemoveContainer" containerID="777cf37702881c8de5d3c35b36577528645dcefdae23876e46b0835ef1a7f9c8" Sep 11 11:57:17.799604 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:17.799585 2566 scope.go:117] "RemoveContainer" containerID="777cf37702881c8de5d3c35b36577528645dcefdae23876e46b0835ef1a7f9c8" Sep 11 11:57:17.799876 ip-10-0-132-170 kubenswrapper[2566]: E0911 11:57:17.799848 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"777cf37702881c8de5d3c35b36577528645dcefdae23876e46b0835ef1a7f9c8\": container with ID starting with 777cf37702881c8de5d3c35b36577528645dcefdae23876e46b0835ef1a7f9c8 not found: ID does not exist" containerID="777cf37702881c8de5d3c35b36577528645dcefdae23876e46b0835ef1a7f9c8" Sep 11 11:57:17.799946 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:17.799918 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"777cf37702881c8de5d3c35b36577528645dcefdae23876e46b0835ef1a7f9c8"} err="failed to get container status \"777cf37702881c8de5d3c35b36577528645dcefdae23876e46b0835ef1a7f9c8\": rpc error: code = NotFound desc = could not find container \"777cf37702881c8de5d3c35b36577528645dcefdae23876e46b0835ef1a7f9c8\": container with ID starting with 777cf37702881c8de5d3c35b36577528645dcefdae23876e46b0835ef1a7f9c8 not found: ID does not exist" Sep 11 11:57:17.817138 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:17.817107 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4r2b9"] Sep 11 11:57:17.821714 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:17.821689 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4r2b9"] Sep 11 11:57:18.757345 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:18.757313 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15a1dd89-5959-4508-aae7-254d5852b8fc" path="/var/lib/kubelet/pods/15a1dd89-5959-4508-aae7-254d5852b8fc/volumes" Sep 11 11:57:33.169754 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.169722 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444"] Sep 11 11:57:33.170180 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.170013 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15a1dd89-5959-4508-aae7-254d5852b8fc" containerName="manager" Sep 11 11:57:33.170180 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.170027 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="15a1dd89-5959-4508-aae7-254d5852b8fc" containerName="manager" Sep 11 11:57:33.170180 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.170091 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="15a1dd89-5959-4508-aae7-254d5852b8fc" containerName="manager" Sep 11 11:57:33.193470 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.193435 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444"] Sep 11 11:57:33.193618 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.193557 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" Sep 11 11:57:33.196091 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.196068 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-rgvr9\"" Sep 11 11:57:33.287621 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.287585 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c888eade-3e00-4c36-b236-6c86b58d33bc-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7v444\" (UID: \"c888eade-3e00-4c36-b236-6c86b58d33bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" Sep 11 11:57:33.287621 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.287622 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87x2t\" (UniqueName: \"kubernetes.io/projected/c888eade-3e00-4c36-b236-6c86b58d33bc-kube-api-access-87x2t\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7v444\" (UID: \"c888eade-3e00-4c36-b236-6c86b58d33bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" Sep 11 11:57:33.287847 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.287651 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c888eade-3e00-4c36-b236-6c86b58d33bc-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7v444\" (UID: \"c888eade-3e00-4c36-b236-6c86b58d33bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" Sep 11 11:57:33.287847 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.287678 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c888eade-3e00-4c36-b236-6c86b58d33bc-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7v444\" (UID: \"c888eade-3e00-4c36-b236-6c86b58d33bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" Sep 11 11:57:33.287847 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.287716 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c888eade-3e00-4c36-b236-6c86b58d33bc-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7v444\" (UID: \"c888eade-3e00-4c36-b236-6c86b58d33bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" Sep 11 11:57:33.287847 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.287778 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c888eade-3e00-4c36-b236-6c86b58d33bc-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7v444\" (UID: \"c888eade-3e00-4c36-b236-6c86b58d33bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" Sep 11 11:57:33.287847 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.287813 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c888eade-3e00-4c36-b236-6c86b58d33bc-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7v444\" (UID: \"c888eade-3e00-4c36-b236-6c86b58d33bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" Sep 11 11:57:33.288061 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.287909 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c888eade-3e00-4c36-b236-6c86b58d33bc-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7v444\" (UID: \"c888eade-3e00-4c36-b236-6c86b58d33bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" Sep 11 11:57:33.288061 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.287952 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c888eade-3e00-4c36-b236-6c86b58d33bc-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7v444\" (UID: \"c888eade-3e00-4c36-b236-6c86b58d33bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" Sep 11 11:57:33.388935 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.388905 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c888eade-3e00-4c36-b236-6c86b58d33bc-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7v444\" (UID: \"c888eade-3e00-4c36-b236-6c86b58d33bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" Sep 11 11:57:33.388935 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.388939 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87x2t\" (UniqueName: \"kubernetes.io/projected/c888eade-3e00-4c36-b236-6c86b58d33bc-kube-api-access-87x2t\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7v444\" (UID: \"c888eade-3e00-4c36-b236-6c86b58d33bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" Sep 11 11:57:33.389179 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.388958 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c888eade-3e00-4c36-b236-6c86b58d33bc-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7v444\" (UID: \"c888eade-3e00-4c36-b236-6c86b58d33bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" Sep 11 11:57:33.389179 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.388977 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c888eade-3e00-4c36-b236-6c86b58d33bc-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7v444\" (UID: \"c888eade-3e00-4c36-b236-6c86b58d33bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" Sep 11 11:57:33.389179 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.389015 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c888eade-3e00-4c36-b236-6c86b58d33bc-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7v444\" (UID: \"c888eade-3e00-4c36-b236-6c86b58d33bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" Sep 11 11:57:33.389179 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.389051 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c888eade-3e00-4c36-b236-6c86b58d33bc-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7v444\" (UID: \"c888eade-3e00-4c36-b236-6c86b58d33bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" Sep 11 11:57:33.389179 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.389079 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c888eade-3e00-4c36-b236-6c86b58d33bc-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7v444\" (UID: \"c888eade-3e00-4c36-b236-6c86b58d33bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" Sep 11 11:57:33.389179 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.389110 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c888eade-3e00-4c36-b236-6c86b58d33bc-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7v444\" (UID: \"c888eade-3e00-4c36-b236-6c86b58d33bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" Sep 11 11:57:33.389179 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.389139 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c888eade-3e00-4c36-b236-6c86b58d33bc-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7v444\" (UID: \"c888eade-3e00-4c36-b236-6c86b58d33bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" Sep 11 11:57:33.389527 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.389394 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c888eade-3e00-4c36-b236-6c86b58d33bc-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7v444\" (UID: \"c888eade-3e00-4c36-b236-6c86b58d33bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" Sep 11 11:57:33.389793 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.389767 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c888eade-3e00-4c36-b236-6c86b58d33bc-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7v444\" (UID: \"c888eade-3e00-4c36-b236-6c86b58d33bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" Sep 11 11:57:33.389905 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.389827 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c888eade-3e00-4c36-b236-6c86b58d33bc-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7v444\" (UID: \"c888eade-3e00-4c36-b236-6c86b58d33bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" Sep 11 11:57:33.389905 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.389892 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c888eade-3e00-4c36-b236-6c86b58d33bc-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7v444\" (UID: \"c888eade-3e00-4c36-b236-6c86b58d33bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" Sep 11 11:57:33.390267 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.390244 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c888eade-3e00-4c36-b236-6c86b58d33bc-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7v444\" (UID: \"c888eade-3e00-4c36-b236-6c86b58d33bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" Sep 11 11:57:33.391341 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.391310 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c888eade-3e00-4c36-b236-6c86b58d33bc-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7v444\" (UID: \"c888eade-3e00-4c36-b236-6c86b58d33bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" Sep 11 11:57:33.391564 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.391548 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c888eade-3e00-4c36-b236-6c86b58d33bc-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7v444\" (UID: \"c888eade-3e00-4c36-b236-6c86b58d33bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" Sep 11 11:57:33.415717 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.415690 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87x2t\" (UniqueName: \"kubernetes.io/projected/c888eade-3e00-4c36-b236-6c86b58d33bc-kube-api-access-87x2t\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7v444\" (UID: \"c888eade-3e00-4c36-b236-6c86b58d33bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" Sep 11 11:57:33.423099 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.423042 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c888eade-3e00-4c36-b236-6c86b58d33bc-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7v444\" (UID: \"c888eade-3e00-4c36-b236-6c86b58d33bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" Sep 11 11:57:33.504349 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.504314 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" Sep 11 11:57:33.782444 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.782369 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444"] Sep 11 11:57:33.785615 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:57:33.785588 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc888eade_3e00_4c36_b236_6c86b58d33bc.slice/crio-66a4af7a04de3351add0067f16c92d4d289842d507971cb0e6c8901fc0537dfd WatchSource:0}: Error finding container 66a4af7a04de3351add0067f16c92d4d289842d507971cb0e6c8901fc0537dfd: Status 404 returned error can't find the container with id 66a4af7a04de3351add0067f16c92d4d289842d507971cb0e6c8901fc0537dfd Sep 11 11:57:33.787691 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.787661 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 11:57:33.787748 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.787724 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 11:57:33.787782 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.787752 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 11:57:33.863954 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.863921 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" event={"ID":"c888eade-3e00-4c36-b236-6c86b58d33bc","Type":"ContainerStarted","Data":"1292579f675aafa7eb1273491b26c63ae7e4ca34e3c659c7a65a1e9830b39d56"} Sep 11 11:57:33.864121 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.863970 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" event={"ID":"c888eade-3e00-4c36-b236-6c86b58d33bc","Type":"ContainerStarted","Data":"66a4af7a04de3351add0067f16c92d4d289842d507971cb0e6c8901fc0537dfd"} Sep 11 11:57:33.890974 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:33.890921 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" podStartSLOduration=0.890904821 podStartE2EDuration="890.904821ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:57:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:57:33.888190855 +0000 UTC m=+585.653164041" watchObservedRunningTime="2026-09-11 11:57:33.890904821 +0000 UTC m=+585.655877993" Sep 11 11:57:34.505500 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:34.505464 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" Sep 11 11:57:34.510090 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:34.510061 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" Sep 11 11:57:34.869077 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:34.869043 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" Sep 11 11:57:34.869912 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:34.869889 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" Sep 11 11:57:36.129404 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:36.129369 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-b92lc"] Sep 11 11:57:36.132818 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:36.132796 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-b92lc" Sep 11 11:57:36.135628 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:36.135609 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-pq2dp\"" Sep 11 11:57:36.142484 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:36.142462 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-b92lc"] Sep 11 11:57:36.215007 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:36.214978 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drr6c\" (UniqueName: \"kubernetes.io/projected/6ef45ec3-3277-43c5-9672-7d876b0fd421-kube-api-access-drr6c\") pod \"authorino-59c974bd48-b92lc\" (UID: \"6ef45ec3-3277-43c5-9672-7d876b0fd421\") " pod="kuadrant-system/authorino-59c974bd48-b92lc" Sep 11 11:57:36.315932 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:36.315882 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drr6c\" (UniqueName: \"kubernetes.io/projected/6ef45ec3-3277-43c5-9672-7d876b0fd421-kube-api-access-drr6c\") pod \"authorino-59c974bd48-b92lc\" (UID: \"6ef45ec3-3277-43c5-9672-7d876b0fd421\") " pod="kuadrant-system/authorino-59c974bd48-b92lc" Sep 11 11:57:36.330014 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:36.329989 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drr6c\" (UniqueName: \"kubernetes.io/projected/6ef45ec3-3277-43c5-9672-7d876b0fd421-kube-api-access-drr6c\") pod \"authorino-59c974bd48-b92lc\" (UID: \"6ef45ec3-3277-43c5-9672-7d876b0fd421\") " pod="kuadrant-system/authorino-59c974bd48-b92lc" Sep 11 11:57:36.441743 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:36.441663 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-b92lc" Sep 11 11:57:36.711497 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:36.711426 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-b92lc"] Sep 11 11:57:36.714474 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:57:36.714451 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6ef45ec3_3277_43c5_9672_7d876b0fd421.slice/crio-d564732aad51b8e94fd5787e6b7c081398f7a3b3c6c743686ff8336fb06b3e9c WatchSource:0}: Error finding container d564732aad51b8e94fd5787e6b7c081398f7a3b3c6c743686ff8336fb06b3e9c: Status 404 returned error can't find the container with id d564732aad51b8e94fd5787e6b7c081398f7a3b3c6c743686ff8336fb06b3e9c Sep 11 11:57:36.878763 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:36.878727 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-b92lc" event={"ID":"6ef45ec3-3277-43c5-9672-7d876b0fd421","Type":"ContainerStarted","Data":"d564732aad51b8e94fd5787e6b7c081398f7a3b3c6c743686ff8336fb06b3e9c"} Sep 11 11:57:38.889750 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:38.889712 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-b92lc" event={"ID":"6ef45ec3-3277-43c5-9672-7d876b0fd421","Type":"ContainerStarted","Data":"6f4e73140e304672f76f5d1d4db6b3a3e3bc5d4f70f3ea03e92dcb24e923f56f"} Sep 11 11:57:38.911820 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:57:38.911767 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-b92lc" podStartSLOduration=1.258431805 podStartE2EDuration="2.91175076s" totalImagesPullingTime="1.653318955s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:57:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:57:38.909618062 +0000 UTC m=+590.674591241" watchObservedRunningTime="2026-09-11 11:57:38.91175076 +0000 UTC m=+590.676723934" Sep 11 11:58:05.867392 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:05.867345 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-b92lc"] Sep 11 11:58:05.868028 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:05.867518 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-b92lc" podUID="6ef45ec3-3277-43c5-9672-7d876b0fd421" containerName="authorino" containerID="cri-o://6f4e73140e304672f76f5d1d4db6b3a3e3bc5d4f70f3ea03e92dcb24e923f56f" gracePeriod=30 Sep 11 11:58:06.000780 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:06.000734 2566 generic.go:356] "Generic (PLEG): container finished" podID="6ef45ec3-3277-43c5-9672-7d876b0fd421" containerID="6f4e73140e304672f76f5d1d4db6b3a3e3bc5d4f70f3ea03e92dcb24e923f56f" exitCode=0 Sep 11 11:58:06.000944 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:06.000802 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-b92lc" event={"ID":"6ef45ec3-3277-43c5-9672-7d876b0fd421","Type":"ContainerDied","Data":"6f4e73140e304672f76f5d1d4db6b3a3e3bc5d4f70f3ea03e92dcb24e923f56f"} Sep 11 11:58:06.107365 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:06.107339 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-b92lc" Sep 11 11:58:06.155066 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:06.154987 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-drr6c\" (UniqueName: \"kubernetes.io/projected/6ef45ec3-3277-43c5-9672-7d876b0fd421-kube-api-access-drr6c\") pod \"6ef45ec3-3277-43c5-9672-7d876b0fd421\" (UID: \"6ef45ec3-3277-43c5-9672-7d876b0fd421\") " Sep 11 11:58:06.157012 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:06.156987 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ef45ec3-3277-43c5-9672-7d876b0fd421-kube-api-access-drr6c" (OuterVolumeSpecName: "kube-api-access-drr6c") pod "6ef45ec3-3277-43c5-9672-7d876b0fd421" (UID: "6ef45ec3-3277-43c5-9672-7d876b0fd421"). InnerVolumeSpecName "kube-api-access-drr6c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 11:58:06.256187 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:06.256151 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drr6c\" (UniqueName: \"kubernetes.io/projected/6ef45ec3-3277-43c5-9672-7d876b0fd421-kube-api-access-drr6c\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 11:58:07.006994 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:07.006955 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-b92lc" event={"ID":"6ef45ec3-3277-43c5-9672-7d876b0fd421","Type":"ContainerDied","Data":"d564732aad51b8e94fd5787e6b7c081398f7a3b3c6c743686ff8336fb06b3e9c"} Sep 11 11:58:07.006994 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:07.006978 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-b92lc" Sep 11 11:58:07.007441 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:07.007010 2566 scope.go:117] "RemoveContainer" containerID="6f4e73140e304672f76f5d1d4db6b3a3e3bc5d4f70f3ea03e92dcb24e923f56f" Sep 11 11:58:07.024446 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:07.024390 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-b92lc"] Sep 11 11:58:07.029183 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:07.029161 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-b92lc"] Sep 11 11:58:08.758295 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:08.758251 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ef45ec3-3277-43c5-9672-7d876b0fd421" path="/var/lib/kubelet/pods/6ef45ec3-3277-43c5-9672-7d876b0fd421/volumes" Sep 11 11:58:10.749614 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:10.749580 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-bc7558999-5wqdq"] Sep 11 11:58:10.750048 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:10.750030 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ef45ec3-3277-43c5-9672-7d876b0fd421" containerName="authorino" Sep 11 11:58:10.750095 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:10.750051 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ef45ec3-3277-43c5-9672-7d876b0fd421" containerName="authorino" Sep 11 11:58:10.750140 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:10.750130 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ef45ec3-3277-43c5-9672-7d876b0fd421" containerName="authorino" Sep 11 11:58:10.755113 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:10.755087 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-bc7558999-5wqdq" Sep 11 11:58:10.757667 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:10.757645 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 11 11:58:10.757775 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:10.757645 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-qmtgj\"" Sep 11 11:58:10.757775 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:10.757734 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 11 11:58:10.758050 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:10.758027 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 11 11:58:10.767351 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:10.767328 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-bc7558999-5wqdq"] Sep 11 11:58:10.795558 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:10.795522 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j97s\" (UniqueName: \"kubernetes.io/projected/7d9d0448-c874-41dd-b616-7366857ddde2-kube-api-access-7j97s\") pod \"maas-controller-bc7558999-5wqdq\" (UID: \"7d9d0448-c874-41dd-b616-7366857ddde2\") " pod="opendatahub/maas-controller-bc7558999-5wqdq" Sep 11 11:58:10.795700 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:10.795564 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7d9d0448-c874-41dd-b616-7366857ddde2-webhook-cert\") pod \"maas-controller-bc7558999-5wqdq\" (UID: \"7d9d0448-c874-41dd-b616-7366857ddde2\") " pod="opendatahub/maas-controller-bc7558999-5wqdq" Sep 11 11:58:10.795700 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:10.795656 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d9d0448-c874-41dd-b616-7366857ddde2-metrics-certs\") pod \"maas-controller-bc7558999-5wqdq\" (UID: \"7d9d0448-c874-41dd-b616-7366857ddde2\") " pod="opendatahub/maas-controller-bc7558999-5wqdq" Sep 11 11:58:10.896441 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:10.896353 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d9d0448-c874-41dd-b616-7366857ddde2-metrics-certs\") pod \"maas-controller-bc7558999-5wqdq\" (UID: \"7d9d0448-c874-41dd-b616-7366857ddde2\") " pod="opendatahub/maas-controller-bc7558999-5wqdq" Sep 11 11:58:10.896441 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:10.896436 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7j97s\" (UniqueName: \"kubernetes.io/projected/7d9d0448-c874-41dd-b616-7366857ddde2-kube-api-access-7j97s\") pod \"maas-controller-bc7558999-5wqdq\" (UID: \"7d9d0448-c874-41dd-b616-7366857ddde2\") " pod="opendatahub/maas-controller-bc7558999-5wqdq" Sep 11 11:58:10.896683 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:10.896485 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7d9d0448-c874-41dd-b616-7366857ddde2-webhook-cert\") pod \"maas-controller-bc7558999-5wqdq\" (UID: \"7d9d0448-c874-41dd-b616-7366857ddde2\") " pod="opendatahub/maas-controller-bc7558999-5wqdq" Sep 11 11:58:10.898762 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:10.898737 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7d9d0448-c874-41dd-b616-7366857ddde2-webhook-cert\") pod \"maas-controller-bc7558999-5wqdq\" (UID: \"7d9d0448-c874-41dd-b616-7366857ddde2\") " pod="opendatahub/maas-controller-bc7558999-5wqdq" Sep 11 11:58:10.898891 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:10.898851 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d9d0448-c874-41dd-b616-7366857ddde2-metrics-certs\") pod \"maas-controller-bc7558999-5wqdq\" (UID: \"7d9d0448-c874-41dd-b616-7366857ddde2\") " pod="opendatahub/maas-controller-bc7558999-5wqdq" Sep 11 11:58:10.907747 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:10.907722 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j97s\" (UniqueName: \"kubernetes.io/projected/7d9d0448-c874-41dd-b616-7366857ddde2-kube-api-access-7j97s\") pod \"maas-controller-bc7558999-5wqdq\" (UID: \"7d9d0448-c874-41dd-b616-7366857ddde2\") " pod="opendatahub/maas-controller-bc7558999-5wqdq" Sep 11 11:58:11.071014 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:11.070983 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-bc7558999-5wqdq" Sep 11 11:58:11.348929 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:11.348899 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-bc7558999-5wqdq"] Sep 11 11:58:11.349753 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:58:11.349727 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d9d0448_c874_41dd_b616_7366857ddde2.slice/crio-01bd0319b7a29df50cefb0dd0d9b6e89758c14a5908e4d2ea64ed255ddcad663 WatchSource:0}: Error finding container 01bd0319b7a29df50cefb0dd0d9b6e89758c14a5908e4d2ea64ed255ddcad663: Status 404 returned error can't find the container with id 01bd0319b7a29df50cefb0dd0d9b6e89758c14a5908e4d2ea64ed255ddcad663 Sep 11 11:58:12.031845 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:12.031791 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-bc7558999-5wqdq" event={"ID":"7d9d0448-c874-41dd-b616-7366857ddde2","Type":"ContainerStarted","Data":"01bd0319b7a29df50cefb0dd0d9b6e89758c14a5908e4d2ea64ed255ddcad663"} Sep 11 11:58:14.043811 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:14.043721 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-bc7558999-5wqdq" event={"ID":"7d9d0448-c874-41dd-b616-7366857ddde2","Type":"ContainerStarted","Data":"e8288194f1e49ec94deb97bf18454129b578e305d6b1fc7bac2e6cbf57cab982"} Sep 11 11:58:14.044186 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:14.043846 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-bc7558999-5wqdq" Sep 11 11:58:14.063844 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:14.063780 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-bc7558999-5wqdq" podStartSLOduration=1.7567066649999998 podStartE2EDuration="4.063736807s" totalImagesPullingTime="2.307030142s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:58:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:58:14.063225249 +0000 UTC m=+625.828198428" watchObservedRunningTime="2026-09-11 11:58:14.063736807 +0000 UTC m=+625.828709983" Sep 11 11:58:24.959904 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:24.959860 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-bc7558999-5wqdq"] Sep 11 11:58:24.960259 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:24.960081 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-bc7558999-5wqdq" podUID="7d9d0448-c874-41dd-b616-7366857ddde2" containerName="manager" containerID="cri-o://e8288194f1e49ec94deb97bf18454129b578e305d6b1fc7bac2e6cbf57cab982" gracePeriod=30 Sep 11 11:58:24.961760 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:24.961725 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-bc7558999-5wqdq" Sep 11 11:58:25.092784 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:25.092750 2566 generic.go:356] "Generic (PLEG): container finished" podID="7d9d0448-c874-41dd-b616-7366857ddde2" containerID="e8288194f1e49ec94deb97bf18454129b578e305d6b1fc7bac2e6cbf57cab982" exitCode=0 Sep 11 11:58:25.092969 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:25.092797 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-bc7558999-5wqdq" event={"ID":"7d9d0448-c874-41dd-b616-7366857ddde2","Type":"ContainerDied","Data":"e8288194f1e49ec94deb97bf18454129b578e305d6b1fc7bac2e6cbf57cab982"} Sep 11 11:58:25.199158 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:25.199129 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-bc7558999-5wqdq" Sep 11 11:58:25.315494 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:25.315465 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7j97s\" (UniqueName: \"kubernetes.io/projected/7d9d0448-c874-41dd-b616-7366857ddde2-kube-api-access-7j97s\") pod \"7d9d0448-c874-41dd-b616-7366857ddde2\" (UID: \"7d9d0448-c874-41dd-b616-7366857ddde2\") " Sep 11 11:58:25.315643 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:25.315510 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7d9d0448-c874-41dd-b616-7366857ddde2-webhook-cert\") pod \"7d9d0448-c874-41dd-b616-7366857ddde2\" (UID: \"7d9d0448-c874-41dd-b616-7366857ddde2\") " Sep 11 11:58:25.315643 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:25.315543 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d9d0448-c874-41dd-b616-7366857ddde2-metrics-certs\") pod \"7d9d0448-c874-41dd-b616-7366857ddde2\" (UID: \"7d9d0448-c874-41dd-b616-7366857ddde2\") " Sep 11 11:58:25.317678 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:25.317643 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d9d0448-c874-41dd-b616-7366857ddde2-kube-api-access-7j97s" (OuterVolumeSpecName: "kube-api-access-7j97s") pod "7d9d0448-c874-41dd-b616-7366857ddde2" (UID: "7d9d0448-c874-41dd-b616-7366857ddde2"). InnerVolumeSpecName "kube-api-access-7j97s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 11:58:25.317796 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:25.317668 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d9d0448-c874-41dd-b616-7366857ddde2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7d9d0448-c874-41dd-b616-7366857ddde2" (UID: "7d9d0448-c874-41dd-b616-7366857ddde2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 11:58:25.317796 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:25.317688 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d9d0448-c874-41dd-b616-7366857ddde2-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "7d9d0448-c874-41dd-b616-7366857ddde2" (UID: "7d9d0448-c874-41dd-b616-7366857ddde2"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 11:58:25.417025 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:25.416973 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7j97s\" (UniqueName: \"kubernetes.io/projected/7d9d0448-c874-41dd-b616-7366857ddde2-kube-api-access-7j97s\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 11:58:25.417025 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:25.417021 2566 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7d9d0448-c874-41dd-b616-7366857ddde2-webhook-cert\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 11:58:25.417025 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:25.417030 2566 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d9d0448-c874-41dd-b616-7366857ddde2-metrics-certs\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 11:58:26.098045 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:26.098010 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-bc7558999-5wqdq" event={"ID":"7d9d0448-c874-41dd-b616-7366857ddde2","Type":"ContainerDied","Data":"01bd0319b7a29df50cefb0dd0d9b6e89758c14a5908e4d2ea64ed255ddcad663"} Sep 11 11:58:26.098045 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:26.098035 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-bc7558999-5wqdq" Sep 11 11:58:26.098548 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:26.098079 2566 scope.go:117] "RemoveContainer" containerID="e8288194f1e49ec94deb97bf18454129b578e305d6b1fc7bac2e6cbf57cab982" Sep 11 11:58:26.123060 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:26.123036 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-bc7558999-5wqdq"] Sep 11 11:58:26.129099 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:26.129074 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-bc7558999-5wqdq"] Sep 11 11:58:26.758446 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:26.758410 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d9d0448-c874-41dd-b616-7366857ddde2" path="/var/lib/kubelet/pods/7d9d0448-c874-41dd-b616-7366857ddde2/volumes" Sep 11 11:58:49.275551 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:49.275510 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-669df8b46-wcvgj"] Sep 11 11:58:49.275973 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:49.275780 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d9d0448-c874-41dd-b616-7366857ddde2" containerName="manager" Sep 11 11:58:49.275973 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:49.275790 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d9d0448-c874-41dd-b616-7366857ddde2" containerName="manager" Sep 11 11:58:49.275973 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:49.275839 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d9d0448-c874-41dd-b616-7366857ddde2" containerName="manager" Sep 11 11:58:49.279497 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:49.279482 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-669df8b46-wcvgj" Sep 11 11:58:49.282056 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:49.282034 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Sep 11 11:58:49.282191 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:49.282096 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 11 11:58:49.282394 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:49.282374 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Sep 11 11:58:49.282488 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:49.282378 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-rrxjh\"" Sep 11 11:58:49.283024 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:49.283004 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Sep 11 11:58:49.293617 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:49.293597 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-669df8b46-wcvgj"] Sep 11 11:58:49.317092 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:49.317059 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f09434f9-9518-46b1-9311-03bfa6265118-maas-api-tls\") pod \"maas-api-669df8b46-wcvgj\" (UID: \"f09434f9-9518-46b1-9311-03bfa6265118\") " pod="odh-ai-gateway-infra/maas-api-669df8b46-wcvgj" Sep 11 11:58:49.317092 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:49.317093 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f09434f9-9518-46b1-9311-03bfa6265118-metrics-certs\") pod \"maas-api-669df8b46-wcvgj\" (UID: \"f09434f9-9518-46b1-9311-03bfa6265118\") " pod="odh-ai-gateway-infra/maas-api-669df8b46-wcvgj" Sep 11 11:58:49.317290 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:49.317118 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jk7n\" (UniqueName: \"kubernetes.io/projected/f09434f9-9518-46b1-9311-03bfa6265118-kube-api-access-4jk7n\") pod \"maas-api-669df8b46-wcvgj\" (UID: \"f09434f9-9518-46b1-9311-03bfa6265118\") " pod="odh-ai-gateway-infra/maas-api-669df8b46-wcvgj" Sep 11 11:58:49.317290 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:49.317144 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f09434f9-9518-46b1-9311-03bfa6265118-openshift-service-ca\") pod \"maas-api-669df8b46-wcvgj\" (UID: \"f09434f9-9518-46b1-9311-03bfa6265118\") " pod="odh-ai-gateway-infra/maas-api-669df8b46-wcvgj" Sep 11 11:58:49.417483 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:49.417446 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f09434f9-9518-46b1-9311-03bfa6265118-maas-api-tls\") pod \"maas-api-669df8b46-wcvgj\" (UID: \"f09434f9-9518-46b1-9311-03bfa6265118\") " pod="odh-ai-gateway-infra/maas-api-669df8b46-wcvgj" Sep 11 11:58:49.417483 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:49.417487 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f09434f9-9518-46b1-9311-03bfa6265118-metrics-certs\") pod \"maas-api-669df8b46-wcvgj\" (UID: \"f09434f9-9518-46b1-9311-03bfa6265118\") " pod="odh-ai-gateway-infra/maas-api-669df8b46-wcvgj" Sep 11 11:58:49.417680 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:49.417600 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jk7n\" (UniqueName: \"kubernetes.io/projected/f09434f9-9518-46b1-9311-03bfa6265118-kube-api-access-4jk7n\") pod \"maas-api-669df8b46-wcvgj\" (UID: \"f09434f9-9518-46b1-9311-03bfa6265118\") " pod="odh-ai-gateway-infra/maas-api-669df8b46-wcvgj" Sep 11 11:58:49.417680 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:49.417647 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f09434f9-9518-46b1-9311-03bfa6265118-openshift-service-ca\") pod \"maas-api-669df8b46-wcvgj\" (UID: \"f09434f9-9518-46b1-9311-03bfa6265118\") " pod="odh-ai-gateway-infra/maas-api-669df8b46-wcvgj" Sep 11 11:58:49.418272 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:49.418253 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f09434f9-9518-46b1-9311-03bfa6265118-openshift-service-ca\") pod \"maas-api-669df8b46-wcvgj\" (UID: \"f09434f9-9518-46b1-9311-03bfa6265118\") " pod="odh-ai-gateway-infra/maas-api-669df8b46-wcvgj" Sep 11 11:58:49.419875 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:49.419847 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f09434f9-9518-46b1-9311-03bfa6265118-metrics-certs\") pod \"maas-api-669df8b46-wcvgj\" (UID: \"f09434f9-9518-46b1-9311-03bfa6265118\") " pod="odh-ai-gateway-infra/maas-api-669df8b46-wcvgj" Sep 11 11:58:49.419931 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:49.419881 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f09434f9-9518-46b1-9311-03bfa6265118-maas-api-tls\") pod \"maas-api-669df8b46-wcvgj\" (UID: \"f09434f9-9518-46b1-9311-03bfa6265118\") " pod="odh-ai-gateway-infra/maas-api-669df8b46-wcvgj" Sep 11 11:58:49.427110 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:49.427083 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jk7n\" (UniqueName: \"kubernetes.io/projected/f09434f9-9518-46b1-9311-03bfa6265118-kube-api-access-4jk7n\") pod \"maas-api-669df8b46-wcvgj\" (UID: \"f09434f9-9518-46b1-9311-03bfa6265118\") " pod="odh-ai-gateway-infra/maas-api-669df8b46-wcvgj" Sep 11 11:58:49.588798 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:49.588763 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-669df8b46-wcvgj" Sep 11 11:58:49.873601 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:49.873522 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-669df8b46-wcvgj"] Sep 11 11:58:49.876606 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:58:49.876580 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf09434f9_9518_46b1_9311_03bfa6265118.slice/crio-cddbb1195c6a15fdaf04879c56e266aa8d713a8134bff39edd8075267fdd5c2d WatchSource:0}: Error finding container cddbb1195c6a15fdaf04879c56e266aa8d713a8134bff39edd8075267fdd5c2d: Status 404 returned error can't find the container with id cddbb1195c6a15fdaf04879c56e266aa8d713a8134bff39edd8075267fdd5c2d Sep 11 11:58:49.906824 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:49.906788 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-pq4hx"] Sep 11 11:58:49.909937 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:49.909918 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-pq4hx" Sep 11 11:58:49.912901 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:49.912880 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Sep 11 11:58:49.913013 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:49.912995 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-nsjb4\"" Sep 11 11:58:49.920718 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:49.920695 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7thfm\" (UniqueName: \"kubernetes.io/projected/e55f3a24-f1c7-4556-a5eb-8a9ba6e89fe1-kube-api-access-7thfm\") pod \"payload-processing-5b54b7fbb7-pq4hx\" (UID: \"e55f3a24-f1c7-4556-a5eb-8a9ba6e89fe1\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-pq4hx" Sep 11 11:58:49.920801 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:49.920731 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e55f3a24-f1c7-4556-a5eb-8a9ba6e89fe1-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-pq4hx\" (UID: \"e55f3a24-f1c7-4556-a5eb-8a9ba6e89fe1\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-pq4hx" Sep 11 11:58:49.925303 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:49.925278 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-pq4hx"] Sep 11 11:58:50.021154 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:50.021122 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7thfm\" (UniqueName: \"kubernetes.io/projected/e55f3a24-f1c7-4556-a5eb-8a9ba6e89fe1-kube-api-access-7thfm\") pod \"payload-processing-5b54b7fbb7-pq4hx\" (UID: \"e55f3a24-f1c7-4556-a5eb-8a9ba6e89fe1\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-pq4hx" Sep 11 11:58:50.021328 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:50.021168 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e55f3a24-f1c7-4556-a5eb-8a9ba6e89fe1-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-pq4hx\" (UID: \"e55f3a24-f1c7-4556-a5eb-8a9ba6e89fe1\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-pq4hx" Sep 11 11:58:50.021905 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:50.021880 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e55f3a24-f1c7-4556-a5eb-8a9ba6e89fe1-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-pq4hx\" (UID: \"e55f3a24-f1c7-4556-a5eb-8a9ba6e89fe1\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-pq4hx" Sep 11 11:58:50.033461 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:50.033434 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7thfm\" (UniqueName: \"kubernetes.io/projected/e55f3a24-f1c7-4556-a5eb-8a9ba6e89fe1-kube-api-access-7thfm\") pod \"payload-processing-5b54b7fbb7-pq4hx\" (UID: \"e55f3a24-f1c7-4556-a5eb-8a9ba6e89fe1\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-pq4hx" Sep 11 11:58:50.197599 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:50.197517 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-669df8b46-wcvgj" event={"ID":"f09434f9-9518-46b1-9311-03bfa6265118","Type":"ContainerStarted","Data":"cddbb1195c6a15fdaf04879c56e266aa8d713a8134bff39edd8075267fdd5c2d"} Sep 11 11:58:50.220783 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:50.220752 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-pq4hx" Sep 11 11:58:50.494535 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:50.494443 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-pq4hx"] Sep 11 11:58:50.500453 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:58:50.499494 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode55f3a24_f1c7_4556_a5eb_8a9ba6e89fe1.slice/crio-97c309384dbe6a7232b3250bc8e1354020a33c361434e5f646097fbf079bdbb2 WatchSource:0}: Error finding container 97c309384dbe6a7232b3250bc8e1354020a33c361434e5f646097fbf079bdbb2: Status 404 returned error can't find the container with id 97c309384dbe6a7232b3250bc8e1354020a33c361434e5f646097fbf079bdbb2 Sep 11 11:58:51.234982 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:51.234899 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-pq4hx" event={"ID":"e55f3a24-f1c7-4556-a5eb-8a9ba6e89fe1","Type":"ContainerStarted","Data":"97c309384dbe6a7232b3250bc8e1354020a33c361434e5f646097fbf079bdbb2"} Sep 11 11:58:53.245552 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:53.245512 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-669df8b46-wcvgj" event={"ID":"f09434f9-9518-46b1-9311-03bfa6265118","Type":"ContainerStarted","Data":"0f7456d79ccb6734312b7a6bbae80d76985e17e9e8c17f4441668118436a7cb7"} Sep 11 11:58:53.246008 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:53.245611 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-669df8b46-wcvgj" Sep 11 11:58:53.247087 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:53.247065 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-pq4hx" event={"ID":"e55f3a24-f1c7-4556-a5eb-8a9ba6e89fe1","Type":"ContainerStarted","Data":"139445f158db02499758f5a000627bbd0eade66661a9daf1a922c931e2b8967b"} Sep 11 11:58:53.247187 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:53.247172 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-pq4hx" Sep 11 11:58:53.273252 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:53.273209 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-669df8b46-wcvgj" podStartSLOduration=1.503427485 podStartE2EDuration="4.273197735s" totalImagesPullingTime="2.76977025s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:58:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:58:53.271975587 +0000 UTC m=+665.036948761" watchObservedRunningTime="2026-09-11 11:58:53.273197735 +0000 UTC m=+665.038170910" Sep 11 11:58:53.292676 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:53.292622 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-pq4hx" podStartSLOduration=2.141088739 podStartE2EDuration="4.292611645s" totalImagesPullingTime="2.151522906s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:58:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:58:53.291294037 +0000 UTC m=+665.056267212" watchObservedRunningTime="2026-09-11 11:58:53.292611645 +0000 UTC m=+665.057584819" Sep 11 11:58:59.257096 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:59.257023 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-669df8b46-wcvgj" Sep 11 11:58:59.272727 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:59.272697 2566 generic.go:356] "Generic (PLEG): container finished" podID="c888eade-3e00-4c36-b236-6c86b58d33bc" containerID="1292579f675aafa7eb1273491b26c63ae7e4ca34e3c659c7a65a1e9830b39d56" exitCode=0 Sep 11 11:58:59.272860 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:59.272765 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" event={"ID":"c888eade-3e00-4c36-b236-6c86b58d33bc","Type":"ContainerDied","Data":"1292579f675aafa7eb1273491b26c63ae7e4ca34e3c659c7a65a1e9830b39d56"} Sep 11 11:58:59.273107 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:59.273094 2566 scope.go:117] "RemoveContainer" containerID="1292579f675aafa7eb1273491b26c63ae7e4ca34e3c659c7a65a1e9830b39d56" Sep 11 11:58:59.274001 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:59.273974 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 11:58:59.274056 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:59.274047 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 11:58:59.274090 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:58:59.274080 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 11:59:00.283883 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:00.283808 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" event={"ID":"c888eade-3e00-4c36-b236-6c86b58d33bc","Type":"ContainerStarted","Data":"70cc65aa29fddbd20e13bc514b6b77fb3f86cd36dc0b7509c585dbf5e293cde2"} Sep 11 11:59:00.507355 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:00.506466 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" Sep 11 11:59:00.507355 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:00.506607 2566 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-7v444 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.23:15021/healthz/ready\": dial tcp 10.132.0.23:15021: connect: connection refused" start-of-body= Sep 11 11:59:00.507355 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:00.506668 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" podUID="c888eade-3e00-4c36-b236-6c86b58d33bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.23:15021/healthz/ready\": dial tcp 10.132.0.23:15021: connect: connection refused" Sep 11 11:59:01.505372 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:01.505323 2566 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-7v444 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.23:15021/healthz/ready\": dial tcp 10.132.0.23:15021: connect: connection refused" start-of-body= Sep 11 11:59:01.505852 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:01.505436 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" podUID="c888eade-3e00-4c36-b236-6c86b58d33bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.23:15021/healthz/ready\": dial tcp 10.132.0.23:15021: connect: connection refused" Sep 11 11:59:02.505933 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:02.505891 2566 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-7v444 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.23:15021/healthz/ready\": dial tcp 10.132.0.23:15021: connect: connection refused" start-of-body= Sep 11 11:59:02.506424 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:02.505976 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" podUID="c888eade-3e00-4c36-b236-6c86b58d33bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.23:15021/healthz/ready\": dial tcp 10.132.0.23:15021: connect: connection refused" Sep 11 11:59:03.505099 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:03.505057 2566 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-7v444 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.23:15021/healthz/ready\": dial tcp 10.132.0.23:15021: connect: connection refused" start-of-body= Sep 11 11:59:03.505383 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:03.505350 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" podUID="c888eade-3e00-4c36-b236-6c86b58d33bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.23:15021/healthz/ready\": dial tcp 10.132.0.23:15021: connect: connection refused" Sep 11 11:59:04.253992 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:04.253952 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-pq4hx" Sep 11 11:59:04.505622 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:04.505530 2566 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-7v444 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.23:15021/healthz/ready\": dial tcp 10.132.0.23:15021: connect: connection refused" start-of-body= Sep 11 11:59:04.505936 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:04.505899 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" podUID="c888eade-3e00-4c36-b236-6c86b58d33bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.23:15021/healthz/ready\": dial tcp 10.132.0.23:15021: connect: connection refused" Sep 11 11:59:04.869523 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:04.869484 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" Sep 11 11:59:05.505332 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:05.505289 2566 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-7v444 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.23:15021/healthz/ready\": dial tcp 10.132.0.23:15021: connect: connection refused" start-of-body= Sep 11 11:59:05.505714 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:05.505379 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" podUID="c888eade-3e00-4c36-b236-6c86b58d33bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.23:15021/healthz/ready\": dial tcp 10.132.0.23:15021: connect: connection refused" Sep 11 11:59:06.508412 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:06.508382 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" Sep 11 11:59:06.509267 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:06.509247 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7v444" Sep 11 11:59:22.536163 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:22.536129 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw"] Sep 11 11:59:22.539293 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:22.539274 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw" Sep 11 11:59:22.542360 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:22.542337 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 11 11:59:22.542515 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:22.542497 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\"" Sep 11 11:59:22.543488 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:22.543466 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 11 11:59:22.543584 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:22.543530 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-fg5rd\"" Sep 11 11:59:22.551798 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:22.551775 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw"] Sep 11 11:59:22.701894 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:22.701844 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/581ca9c6-c1f4-474c-82a1-51bf6ddd6822-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw\" (UID: \"581ca9c6-c1f4-474c-82a1-51bf6ddd6822\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw" Sep 11 11:59:22.702059 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:22.702000 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/581ca9c6-c1f4-474c-82a1-51bf6ddd6822-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw\" (UID: \"581ca9c6-c1f4-474c-82a1-51bf6ddd6822\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw" Sep 11 11:59:22.702147 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:22.702065 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9t2h\" (UniqueName: \"kubernetes.io/projected/581ca9c6-c1f4-474c-82a1-51bf6ddd6822-kube-api-access-p9t2h\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw\" (UID: \"581ca9c6-c1f4-474c-82a1-51bf6ddd6822\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw" Sep 11 11:59:22.702147 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:22.702136 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/581ca9c6-c1f4-474c-82a1-51bf6ddd6822-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw\" (UID: \"581ca9c6-c1f4-474c-82a1-51bf6ddd6822\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw" Sep 11 11:59:22.702253 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:22.702168 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/581ca9c6-c1f4-474c-82a1-51bf6ddd6822-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw\" (UID: \"581ca9c6-c1f4-474c-82a1-51bf6ddd6822\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw" Sep 11 11:59:22.702253 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:22.702195 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/581ca9c6-c1f4-474c-82a1-51bf6ddd6822-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw\" (UID: \"581ca9c6-c1f4-474c-82a1-51bf6ddd6822\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw" Sep 11 11:59:22.803094 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:22.803000 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/581ca9c6-c1f4-474c-82a1-51bf6ddd6822-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw\" (UID: \"581ca9c6-c1f4-474c-82a1-51bf6ddd6822\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw" Sep 11 11:59:22.803094 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:22.803041 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/581ca9c6-c1f4-474c-82a1-51bf6ddd6822-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw\" (UID: \"581ca9c6-c1f4-474c-82a1-51bf6ddd6822\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw" Sep 11 11:59:22.803094 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:22.803059 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/581ca9c6-c1f4-474c-82a1-51bf6ddd6822-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw\" (UID: \"581ca9c6-c1f4-474c-82a1-51bf6ddd6822\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw" Sep 11 11:59:22.803094 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:22.803097 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/581ca9c6-c1f4-474c-82a1-51bf6ddd6822-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw\" (UID: \"581ca9c6-c1f4-474c-82a1-51bf6ddd6822\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw" Sep 11 11:59:22.803428 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:22.803219 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/581ca9c6-c1f4-474c-82a1-51bf6ddd6822-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw\" (UID: \"581ca9c6-c1f4-474c-82a1-51bf6ddd6822\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw" Sep 11 11:59:22.803428 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:22.803297 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9t2h\" (UniqueName: \"kubernetes.io/projected/581ca9c6-c1f4-474c-82a1-51bf6ddd6822-kube-api-access-p9t2h\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw\" (UID: \"581ca9c6-c1f4-474c-82a1-51bf6ddd6822\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw" Sep 11 11:59:22.803533 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:22.803425 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/581ca9c6-c1f4-474c-82a1-51bf6ddd6822-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw\" (UID: \"581ca9c6-c1f4-474c-82a1-51bf6ddd6822\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw" Sep 11 11:59:22.803533 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:22.803477 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/581ca9c6-c1f4-474c-82a1-51bf6ddd6822-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw\" (UID: \"581ca9c6-c1f4-474c-82a1-51bf6ddd6822\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw" Sep 11 11:59:22.803610 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:22.803546 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/581ca9c6-c1f4-474c-82a1-51bf6ddd6822-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw\" (UID: \"581ca9c6-c1f4-474c-82a1-51bf6ddd6822\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw" Sep 11 11:59:22.805429 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:22.805400 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/581ca9c6-c1f4-474c-82a1-51bf6ddd6822-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw\" (UID: \"581ca9c6-c1f4-474c-82a1-51bf6ddd6822\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw" Sep 11 11:59:22.805578 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:22.805560 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/581ca9c6-c1f4-474c-82a1-51bf6ddd6822-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw\" (UID: \"581ca9c6-c1f4-474c-82a1-51bf6ddd6822\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw" Sep 11 11:59:22.812028 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:22.812003 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9t2h\" (UniqueName: \"kubernetes.io/projected/581ca9c6-c1f4-474c-82a1-51bf6ddd6822-kube-api-access-p9t2h\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw\" (UID: \"581ca9c6-c1f4-474c-82a1-51bf6ddd6822\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw" Sep 11 11:59:22.848697 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:22.848665 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw" Sep 11 11:59:23.120672 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:23.120640 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw"] Sep 11 11:59:23.123584 ip-10-0-132-170 kubenswrapper[2566]: W0911 11:59:23.123551 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod581ca9c6_c1f4_474c_82a1_51bf6ddd6822.slice/crio-f6a93e3b7e50a1088d1cbc8026a6d7acae1b1ef8bc8a42e14eb6ccb0127426d4 WatchSource:0}: Error finding container f6a93e3b7e50a1088d1cbc8026a6d7acae1b1ef8bc8a42e14eb6ccb0127426d4: Status 404 returned error can't find the container with id f6a93e3b7e50a1088d1cbc8026a6d7acae1b1ef8bc8a42e14eb6ccb0127426d4 Sep 11 11:59:23.397358 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:23.397269 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw" event={"ID":"581ca9c6-c1f4-474c-82a1-51bf6ddd6822","Type":"ContainerStarted","Data":"f6a93e3b7e50a1088d1cbc8026a6d7acae1b1ef8bc8a42e14eb6ccb0127426d4"} Sep 11 11:59:24.404637 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:24.404537 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw" event={"ID":"581ca9c6-c1f4-474c-82a1-51bf6ddd6822","Type":"ContainerStarted","Data":"6b1346b63f08c8afc8e0b910ee8c98f4f710a529dab721ebd48476399575aab4"} Sep 11 11:59:24.426365 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:24.426301 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw" podStartSLOduration=1.456069049 podStartE2EDuration="2.4262776s" totalImagesPullingTime="970.208551ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:59:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:59:24.424734585 +0000 UTC m=+696.189707759" watchObservedRunningTime="2026-09-11 11:59:24.4262776 +0000 UTC m=+696.191250778" Sep 11 11:59:32.849778 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:32.849736 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw" Sep 11 11:59:32.850330 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:32.849807 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw" Sep 11 11:59:32.862229 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:32.862199 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw" Sep 11 11:59:33.461624 ip-10-0-132-170 kubenswrapper[2566]: I0911 11:59:33.461597 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw" Sep 11 12:00:32.564816 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:00:32.564771 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-gjc9l"] Sep 11 12:00:32.568432 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:00:32.568411 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gjc9l" Sep 11 12:00:32.572380 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:00:32.572357 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Sep 11 12:00:32.580985 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:00:32.580964 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-gjc9l"] Sep 11 12:00:32.589767 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:00:32.589743 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85wd7\" (UniqueName: \"kubernetes.io/projected/1e88b32b-bc59-44f4-a542-41567be1ada7-kube-api-access-85wd7\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gjc9l\" (UID: \"1e88b32b-bc59-44f4-a542-41567be1ada7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gjc9l" Sep 11 12:00:32.589894 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:00:32.589775 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e88b32b-bc59-44f4-a542-41567be1ada7-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gjc9l\" (UID: \"1e88b32b-bc59-44f4-a542-41567be1ada7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gjc9l" Sep 11 12:00:32.589894 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:00:32.589792 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e88b32b-bc59-44f4-a542-41567be1ada7-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gjc9l\" (UID: \"1e88b32b-bc59-44f4-a542-41567be1ada7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gjc9l" Sep 11 12:00:32.589894 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:00:32.589808 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e88b32b-bc59-44f4-a542-41567be1ada7-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gjc9l\" (UID: \"1e88b32b-bc59-44f4-a542-41567be1ada7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gjc9l" Sep 11 12:00:32.589894 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:00:32.589826 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e88b32b-bc59-44f4-a542-41567be1ada7-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gjc9l\" (UID: \"1e88b32b-bc59-44f4-a542-41567be1ada7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gjc9l" Sep 11 12:00:32.590114 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:00:32.589920 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e88b32b-bc59-44f4-a542-41567be1ada7-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gjc9l\" (UID: \"1e88b32b-bc59-44f4-a542-41567be1ada7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gjc9l" Sep 11 12:00:32.690360 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:00:32.690324 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85wd7\" (UniqueName: \"kubernetes.io/projected/1e88b32b-bc59-44f4-a542-41567be1ada7-kube-api-access-85wd7\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gjc9l\" (UID: \"1e88b32b-bc59-44f4-a542-41567be1ada7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gjc9l" Sep 11 12:00:32.690360 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:00:32.690361 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e88b32b-bc59-44f4-a542-41567be1ada7-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gjc9l\" (UID: \"1e88b32b-bc59-44f4-a542-41567be1ada7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gjc9l" Sep 11 12:00:32.690607 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:00:32.690381 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e88b32b-bc59-44f4-a542-41567be1ada7-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gjc9l\" (UID: \"1e88b32b-bc59-44f4-a542-41567be1ada7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gjc9l" Sep 11 12:00:32.690607 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:00:32.690399 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e88b32b-bc59-44f4-a542-41567be1ada7-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gjc9l\" (UID: \"1e88b32b-bc59-44f4-a542-41567be1ada7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gjc9l" Sep 11 12:00:32.690607 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:00:32.690416 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e88b32b-bc59-44f4-a542-41567be1ada7-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gjc9l\" (UID: \"1e88b32b-bc59-44f4-a542-41567be1ada7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gjc9l" Sep 11 12:00:32.690607 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:00:32.690461 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e88b32b-bc59-44f4-a542-41567be1ada7-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gjc9l\" (UID: \"1e88b32b-bc59-44f4-a542-41567be1ada7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gjc9l" Sep 11 12:00:32.690826 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:00:32.690806 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e88b32b-bc59-44f4-a542-41567be1ada7-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gjc9l\" (UID: \"1e88b32b-bc59-44f4-a542-41567be1ada7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gjc9l" Sep 11 12:00:32.690900 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:00:32.690847 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e88b32b-bc59-44f4-a542-41567be1ada7-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gjc9l\" (UID: \"1e88b32b-bc59-44f4-a542-41567be1ada7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gjc9l" Sep 11 12:00:32.690946 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:00:32.690913 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e88b32b-bc59-44f4-a542-41567be1ada7-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gjc9l\" (UID: \"1e88b32b-bc59-44f4-a542-41567be1ada7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gjc9l" Sep 11 12:00:32.692782 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:00:32.692759 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e88b32b-bc59-44f4-a542-41567be1ada7-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gjc9l\" (UID: \"1e88b32b-bc59-44f4-a542-41567be1ada7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gjc9l" Sep 11 12:00:32.692930 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:00:32.692913 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e88b32b-bc59-44f4-a542-41567be1ada7-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gjc9l\" (UID: \"1e88b32b-bc59-44f4-a542-41567be1ada7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gjc9l" Sep 11 12:00:32.703445 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:00:32.703409 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85wd7\" (UniqueName: \"kubernetes.io/projected/1e88b32b-bc59-44f4-a542-41567be1ada7-kube-api-access-85wd7\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gjc9l\" (UID: \"1e88b32b-bc59-44f4-a542-41567be1ada7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gjc9l" Sep 11 12:00:32.878440 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:00:32.878409 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gjc9l" Sep 11 12:00:33.164169 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:00:33.164089 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-gjc9l"] Sep 11 12:00:33.167376 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:00:33.167341 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e88b32b_bc59_44f4_a542_41567be1ada7.slice/crio-30c9634b514057f099d298e1fceda2fba229af5bfbe7048f46cc8d06a6f5763f WatchSource:0}: Error finding container 30c9634b514057f099d298e1fceda2fba229af5bfbe7048f46cc8d06a6f5763f: Status 404 returned error can't find the container with id 30c9634b514057f099d298e1fceda2fba229af5bfbe7048f46cc8d06a6f5763f Sep 11 12:00:33.168701 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:00:33.168680 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 11 12:00:33.723211 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:00:33.723132 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gjc9l" event={"ID":"1e88b32b-bc59-44f4-a542-41567be1ada7","Type":"ContainerStarted","Data":"9d1698e38bdfd8c58a793e0f8ebe6e857e2f6ac5a04e4d24af0a9fc3d93f758d"} Sep 11 12:00:33.723211 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:00:33.723178 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gjc9l" event={"ID":"1e88b32b-bc59-44f4-a542-41567be1ada7","Type":"ContainerStarted","Data":"30c9634b514057f099d298e1fceda2fba229af5bfbe7048f46cc8d06a6f5763f"} Sep 11 12:00:33.747685 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:00:33.747629 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gjc9l" podStartSLOduration=1.53868261 podStartE2EDuration="1.747613844s" totalImagesPullingTime="208.931234ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:00:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:00:33.745859993 +0000 UTC m=+765.510833170" watchObservedRunningTime="2026-09-11 12:00:33.747613844 +0000 UTC m=+765.512587019" Sep 11 12:00:42.878912 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:00:42.878882 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gjc9l" Sep 11 12:00:42.879299 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:00:42.878939 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gjc9l" Sep 11 12:00:42.891174 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:00:42.891151 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gjc9l" Sep 11 12:00:43.774575 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:00:43.774545 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gjc9l" Sep 11 12:02:08.625985 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:08.625955 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2"] Sep 11 12:02:08.629385 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:08.629360 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" Sep 11 12:02:08.631836 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:08.631816 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-cbc380-openshift-default-dockercfg-5j55r\"" Sep 11 12:02:08.654252 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:08.654227 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2"] Sep 11 12:02:08.695420 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:08.695390 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbp6v\" (UniqueName: \"kubernetes.io/projected/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-kube-api-access-vbp6v\") pod \"e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2\" (UID: \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\") " pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" Sep 11 12:02:08.695579 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:08.695428 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-workload-socket\") pod \"e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2\" (UID: \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\") " pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" Sep 11 12:02:08.695579 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:08.695467 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-istio-data\") pod \"e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2\" (UID: \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\") " pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" Sep 11 12:02:08.695579 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:08.695498 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-workload-certs\") pod \"e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2\" (UID: \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\") " pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" Sep 11 12:02:08.695579 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:08.695566 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-istiod-ca-cert\") pod \"e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2\" (UID: \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\") " pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" Sep 11 12:02:08.695795 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:08.695647 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-istio-podinfo\") pod \"e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2\" (UID: \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\") " pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" Sep 11 12:02:08.695795 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:08.695677 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-istio-envoy\") pod \"e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2\" (UID: \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\") " pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" Sep 11 12:02:08.695795 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:08.695708 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-istio-token\") pod \"e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2\" (UID: \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\") " pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" Sep 11 12:02:08.695795 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:08.695768 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-credential-socket\") pod \"e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2\" (UID: \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\") " pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" Sep 11 12:02:08.796422 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:08.796385 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-istio-data\") pod \"e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2\" (UID: \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\") " pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" Sep 11 12:02:08.796422 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:08.796422 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-workload-certs\") pod \"e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2\" (UID: \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\") " pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" Sep 11 12:02:08.796645 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:08.796441 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-istiod-ca-cert\") pod \"e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2\" (UID: \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\") " pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" Sep 11 12:02:08.796645 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:08.796477 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-istio-podinfo\") pod \"e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2\" (UID: \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\") " pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" Sep 11 12:02:08.796645 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:08.796494 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-istio-envoy\") pod \"e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2\" (UID: \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\") " pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" Sep 11 12:02:08.796645 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:08.796513 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-istio-token\") pod \"e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2\" (UID: \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\") " pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" Sep 11 12:02:08.796645 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:08.796536 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-credential-socket\") pod \"e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2\" (UID: \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\") " pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" Sep 11 12:02:08.796645 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:08.796573 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbp6v\" (UniqueName: \"kubernetes.io/projected/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-kube-api-access-vbp6v\") pod \"e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2\" (UID: \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\") " pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" Sep 11 12:02:08.796983 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:08.796683 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-workload-socket\") pod \"e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2\" (UID: \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\") " pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" Sep 11 12:02:08.797318 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:08.797277 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-credential-socket\") pod \"e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2\" (UID: \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\") " pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" Sep 11 12:02:08.797604 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:08.797579 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-istio-data\") pod \"e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2\" (UID: \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\") " pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" Sep 11 12:02:08.797740 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:08.797629 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-workload-socket\") pod \"e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2\" (UID: \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\") " pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" Sep 11 12:02:08.797843 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:08.797739 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-workload-certs\") pod \"e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2\" (UID: \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\") " pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" Sep 11 12:02:08.800375 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:08.797966 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-istiod-ca-cert\") pod \"e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2\" (UID: \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\") " pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" Sep 11 12:02:08.800375 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:08.799729 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-istio-envoy\") pod \"e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2\" (UID: \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\") " pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" Sep 11 12:02:08.805092 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:08.805070 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-istio-podinfo\") pod \"e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2\" (UID: \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\") " pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" Sep 11 12:02:08.808328 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:08.808306 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-istio-token\") pod \"e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2\" (UID: \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\") " pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" Sep 11 12:02:08.808423 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:08.808353 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbp6v\" (UniqueName: \"kubernetes.io/projected/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-kube-api-access-vbp6v\") pod \"e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2\" (UID: \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\") " pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" Sep 11 12:02:08.939385 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:08.939302 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" Sep 11 12:02:09.226596 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:09.226565 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2"] Sep 11 12:02:09.228543 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:02:09.228515 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67bee4aa_84b4_4138_9ecd_51edc10f2e1a.slice/crio-d7a41fc4156ed956460359dff7a9f08f7fd8a0137462f044e37ce088309baed7 WatchSource:0}: Error finding container d7a41fc4156ed956460359dff7a9f08f7fd8a0137462f044e37ce088309baed7: Status 404 returned error can't find the container with id d7a41fc4156ed956460359dff7a9f08f7fd8a0137462f044e37ce088309baed7 Sep 11 12:02:09.230676 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:09.230642 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:02:09.230739 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:09.230707 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:02:09.230778 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:09.230736 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:02:10.139132 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:10.139094 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" event={"ID":"67bee4aa-84b4-4138-9ecd-51edc10f2e1a","Type":"ContainerStarted","Data":"abf3978897df7df56370bfba52738804adc222b5f91c8eaa1d9fb20b24575342"} Sep 11 12:02:10.139611 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:10.139155 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" event={"ID":"67bee4aa-84b4-4138-9ecd-51edc10f2e1a","Type":"ContainerStarted","Data":"d7a41fc4156ed956460359dff7a9f08f7fd8a0137462f044e37ce088309baed7"} Sep 11 12:02:10.166380 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:10.166331 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" podStartSLOduration=2.166313866 podStartE2EDuration="2.166313866s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:02:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:02:10.165011501 +0000 UTC m=+861.929984731" watchObservedRunningTime="2026-09-11 12:02:10.166313866 +0000 UTC m=+861.931287085" Sep 11 12:02:10.940046 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:10.940006 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" Sep 11 12:02:10.944930 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:10.944906 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" Sep 11 12:02:11.143472 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:11.143439 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" Sep 11 12:02:11.144338 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:11.144317 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" Sep 11 12:02:13.354695 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:13.354655 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9"] Sep 11 12:02:13.358616 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:13.358593 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9" Sep 11 12:02:13.361484 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:13.361463 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-cbc380-serving-cert\"" Sep 11 12:02:13.370634 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:13.370605 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9"] Sep 11 12:02:13.457909 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:13.457853 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58351aff-8939-4449-9523-e07001ad0840-maas-api-tls\") pod \"maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9\" (UID: \"58351aff-8939-4449-9523-e07001ad0840\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9" Sep 11 12:02:13.458089 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:13.457984 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjmgd\" (UniqueName: \"kubernetes.io/projected/58351aff-8939-4449-9523-e07001ad0840-kube-api-access-cjmgd\") pod \"maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9\" (UID: \"58351aff-8939-4449-9523-e07001ad0840\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9" Sep 11 12:02:13.458089 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:13.458023 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58351aff-8939-4449-9523-e07001ad0840-openshift-service-ca\") pod \"maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9\" (UID: \"58351aff-8939-4449-9523-e07001ad0840\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9" Sep 11 12:02:13.458089 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:13.458070 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58351aff-8939-4449-9523-e07001ad0840-metrics-certs\") pod \"maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9\" (UID: \"58351aff-8939-4449-9523-e07001ad0840\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9" Sep 11 12:02:13.559503 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:13.559468 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58351aff-8939-4449-9523-e07001ad0840-maas-api-tls\") pod \"maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9\" (UID: \"58351aff-8939-4449-9523-e07001ad0840\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9" Sep 11 12:02:13.559655 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:13.559521 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjmgd\" (UniqueName: \"kubernetes.io/projected/58351aff-8939-4449-9523-e07001ad0840-kube-api-access-cjmgd\") pod \"maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9\" (UID: \"58351aff-8939-4449-9523-e07001ad0840\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9" Sep 11 12:02:13.559655 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:13.559542 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58351aff-8939-4449-9523-e07001ad0840-openshift-service-ca\") pod \"maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9\" (UID: \"58351aff-8939-4449-9523-e07001ad0840\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9" Sep 11 12:02:13.559655 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:13.559569 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58351aff-8939-4449-9523-e07001ad0840-metrics-certs\") pod \"maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9\" (UID: \"58351aff-8939-4449-9523-e07001ad0840\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9" Sep 11 12:02:13.559655 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:02:13.559616 2566 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-cbc380-serving-cert: secret "maas-api-e2e-inf-a-cbc380-serving-cert" not found Sep 11 12:02:13.559783 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:02:13.559683 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58351aff-8939-4449-9523-e07001ad0840-maas-api-tls podName:58351aff-8939-4449-9523-e07001ad0840 nodeName:}" failed. No retries permitted until 2026-09-11 12:02:14.059666408 +0000 UTC m=+865.824639561 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/58351aff-8939-4449-9523-e07001ad0840-maas-api-tls") pod "maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9" (UID: "58351aff-8939-4449-9523-e07001ad0840") : secret "maas-api-e2e-inf-a-cbc380-serving-cert" not found Sep 11 12:02:13.560377 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:13.560353 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58351aff-8939-4449-9523-e07001ad0840-openshift-service-ca\") pod \"maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9\" (UID: \"58351aff-8939-4449-9523-e07001ad0840\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9" Sep 11 12:02:13.562045 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:13.562028 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58351aff-8939-4449-9523-e07001ad0840-metrics-certs\") pod \"maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9\" (UID: \"58351aff-8939-4449-9523-e07001ad0840\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9" Sep 11 12:02:13.576183 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:13.576160 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjmgd\" (UniqueName: \"kubernetes.io/projected/58351aff-8939-4449-9523-e07001ad0840-kube-api-access-cjmgd\") pod \"maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9\" (UID: \"58351aff-8939-4449-9523-e07001ad0840\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9" Sep 11 12:02:14.065096 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:14.065060 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58351aff-8939-4449-9523-e07001ad0840-maas-api-tls\") pod \"maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9\" (UID: \"58351aff-8939-4449-9523-e07001ad0840\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9" Sep 11 12:02:14.067409 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:14.067388 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58351aff-8939-4449-9523-e07001ad0840-maas-api-tls\") pod \"maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9\" (UID: \"58351aff-8939-4449-9523-e07001ad0840\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9" Sep 11 12:02:14.256726 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:14.256694 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-cbc380-7b4d557d8b-52rrl"] Sep 11 12:02:14.260089 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:14.260069 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cbc380-7b4d557d8b-52rrl" Sep 11 12:02:14.262493 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:14.262471 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-cbc380-dockercfg-9rdhw\"" Sep 11 12:02:14.262598 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:14.262474 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-cbc380\"" Sep 11 12:02:14.269733 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:14.269706 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-cbc380-7b4d557d8b-52rrl"] Sep 11 12:02:14.273232 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:14.273210 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9" Sep 11 12:02:14.367751 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:14.367708 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fcpb\" (UniqueName: \"kubernetes.io/projected/431e293a-0b82-4210-aa0d-e7ed0bab4d00-kube-api-access-8fcpb\") pod \"payload-pre-processing-e2e-inf-a-cbc380-7b4d557d8b-52rrl\" (UID: \"431e293a-0b82-4210-aa0d-e7ed0bab4d00\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cbc380-7b4d557d8b-52rrl" Sep 11 12:02:14.368116 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:14.367833 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/431e293a-0b82-4210-aa0d-e7ed0bab4d00-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-cbc380-7b4d557d8b-52rrl\" (UID: \"431e293a-0b82-4210-aa0d-e7ed0bab4d00\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cbc380-7b4d557d8b-52rrl" Sep 11 12:02:14.468628 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:14.468592 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/431e293a-0b82-4210-aa0d-e7ed0bab4d00-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-cbc380-7b4d557d8b-52rrl\" (UID: \"431e293a-0b82-4210-aa0d-e7ed0bab4d00\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cbc380-7b4d557d8b-52rrl" Sep 11 12:02:14.468796 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:14.468681 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fcpb\" (UniqueName: \"kubernetes.io/projected/431e293a-0b82-4210-aa0d-e7ed0bab4d00-kube-api-access-8fcpb\") pod \"payload-pre-processing-e2e-inf-a-cbc380-7b4d557d8b-52rrl\" (UID: \"431e293a-0b82-4210-aa0d-e7ed0bab4d00\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cbc380-7b4d557d8b-52rrl" Sep 11 12:02:14.469406 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:14.469387 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/431e293a-0b82-4210-aa0d-e7ed0bab4d00-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-cbc380-7b4d557d8b-52rrl\" (UID: \"431e293a-0b82-4210-aa0d-e7ed0bab4d00\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cbc380-7b4d557d8b-52rrl" Sep 11 12:02:14.492854 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:14.492826 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fcpb\" (UniqueName: \"kubernetes.io/projected/431e293a-0b82-4210-aa0d-e7ed0bab4d00-kube-api-access-8fcpb\") pod \"payload-pre-processing-e2e-inf-a-cbc380-7b4d557d8b-52rrl\" (UID: \"431e293a-0b82-4210-aa0d-e7ed0bab4d00\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cbc380-7b4d557d8b-52rrl" Sep 11 12:02:14.563138 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:14.563107 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9"] Sep 11 12:02:14.564386 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:02:14.564354 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58351aff_8939_4449_9523_e07001ad0840.slice/crio-293b205b59864b8cd4100bb65ebb0c68a2b0441558aed22b0a4dbfb51f0c2422 WatchSource:0}: Error finding container 293b205b59864b8cd4100bb65ebb0c68a2b0441558aed22b0a4dbfb51f0c2422: Status 404 returned error can't find the container with id 293b205b59864b8cd4100bb65ebb0c68a2b0441558aed22b0a4dbfb51f0c2422 Sep 11 12:02:14.570615 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:14.570595 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cbc380-7b4d557d8b-52rrl" Sep 11 12:02:14.845534 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:14.845510 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-cbc380-7b4d557d8b-52rrl"] Sep 11 12:02:14.847391 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:02:14.847356 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod431e293a_0b82_4210_aa0d_e7ed0bab4d00.slice/crio-9acfd064c597d58a07ab6ae946d10c18ba28df2e6231f44f9198ba5ab6f150ed WatchSource:0}: Error finding container 9acfd064c597d58a07ab6ae946d10c18ba28df2e6231f44f9198ba5ab6f150ed: Status 404 returned error can't find the container with id 9acfd064c597d58a07ab6ae946d10c18ba28df2e6231f44f9198ba5ab6f150ed Sep 11 12:02:15.163345 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:15.163243 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cbc380-7b4d557d8b-52rrl" event={"ID":"431e293a-0b82-4210-aa0d-e7ed0bab4d00","Type":"ContainerStarted","Data":"21181a6c7d819c51bc5f3f76894f876fbf5eeb67818119b4ec73fa2d45e46271"} Sep 11 12:02:15.163501 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:15.163346 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cbc380-7b4d557d8b-52rrl" Sep 11 12:02:15.163501 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:15.163369 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cbc380-7b4d557d8b-52rrl" event={"ID":"431e293a-0b82-4210-aa0d-e7ed0bab4d00","Type":"ContainerStarted","Data":"9acfd064c597d58a07ab6ae946d10c18ba28df2e6231f44f9198ba5ab6f150ed"} Sep 11 12:02:15.164923 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:15.164897 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9" event={"ID":"58351aff-8939-4449-9523-e07001ad0840","Type":"ContainerStarted","Data":"cc54ae46fd72abf3a4fa7bd4009de713a9c29d58a4dabe56063feefce6b03655"} Sep 11 12:02:15.165064 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:15.164936 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9" event={"ID":"58351aff-8939-4449-9523-e07001ad0840","Type":"ContainerStarted","Data":"293b205b59864b8cd4100bb65ebb0c68a2b0441558aed22b0a4dbfb51f0c2422"} Sep 11 12:02:15.165064 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:15.164960 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9" Sep 11 12:02:15.190432 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:15.190377 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cbc380-7b4d557d8b-52rrl" podStartSLOduration=1.190363115 podStartE2EDuration="1.190363115s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:02:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:02:15.18854476 +0000 UTC m=+866.953517960" watchObservedRunningTime="2026-09-11 12:02:15.190363115 +0000 UTC m=+866.955336290" Sep 11 12:02:15.209117 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:15.209060 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9" podStartSLOduration=1.803737337 podStartE2EDuration="2.209040439s" totalImagesPullingTime="405.303102ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:02:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:02:15.207714807 +0000 UTC m=+866.972687976" watchObservedRunningTime="2026-09-11 12:02:15.209040439 +0000 UTC m=+866.974013616" Sep 11 12:02:17.962604 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:17.962568 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc"] Sep 11 12:02:17.966091 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:17.966069 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" Sep 11 12:02:17.975941 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:17.975922 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-cdec04-openshift-default-dockercfg-6dgv5\"" Sep 11 12:02:18.084152 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:18.084120 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc"] Sep 11 12:02:18.101774 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:18.101733 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-credential-socket\") pod \"e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc\" (UID: \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\") " pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" Sep 11 12:02:18.101971 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:18.101791 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-istio-podinfo\") pod \"e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc\" (UID: \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\") " pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" Sep 11 12:02:18.101971 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:18.101829 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-istio-envoy\") pod \"e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc\" (UID: \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\") " pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" Sep 11 12:02:18.102068 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:18.101973 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-workload-certs\") pod \"e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc\" (UID: \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\") " pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" Sep 11 12:02:18.102068 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:18.102042 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-workload-socket\") pod \"e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc\" (UID: \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\") " pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" Sep 11 12:02:18.102147 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:18.102069 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8p4c\" (UniqueName: \"kubernetes.io/projected/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-kube-api-access-j8p4c\") pod \"e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc\" (UID: \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\") " pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" Sep 11 12:02:18.102147 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:18.102119 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-istio-data\") pod \"e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc\" (UID: \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\") " pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" Sep 11 12:02:18.102221 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:18.102150 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-istio-token\") pod \"e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc\" (UID: \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\") " pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" Sep 11 12:02:18.102221 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:18.102171 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-istiod-ca-cert\") pod \"e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc\" (UID: \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\") " pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" Sep 11 12:02:18.202760 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:18.202722 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-istio-podinfo\") pod \"e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc\" (UID: \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\") " pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" Sep 11 12:02:18.202760 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:18.202762 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-istio-envoy\") pod \"e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc\" (UID: \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\") " pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" Sep 11 12:02:18.203030 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:18.202786 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-workload-certs\") pod \"e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc\" (UID: \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\") " pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" Sep 11 12:02:18.203030 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:18.202823 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-workload-socket\") pod \"e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc\" (UID: \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\") " pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" Sep 11 12:02:18.203030 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:18.202849 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8p4c\" (UniqueName: \"kubernetes.io/projected/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-kube-api-access-j8p4c\") pod \"e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc\" (UID: \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\") " pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" Sep 11 12:02:18.203030 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:18.202913 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-istio-data\") pod \"e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc\" (UID: \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\") " pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" Sep 11 12:02:18.203030 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:18.202942 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-istio-token\") pod \"e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc\" (UID: \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\") " pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" Sep 11 12:02:18.203030 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:18.202970 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-istiod-ca-cert\") pod \"e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc\" (UID: \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\") " pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" Sep 11 12:02:18.203323 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:18.203042 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-credential-socket\") pod \"e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc\" (UID: \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\") " pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" Sep 11 12:02:18.203323 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:18.203267 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-workload-socket\") pod \"e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc\" (UID: \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\") " pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" Sep 11 12:02:18.203493 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:18.203326 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-credential-socket\") pod \"e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc\" (UID: \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\") " pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" Sep 11 12:02:18.203555 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:18.203512 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-istio-data\") pod \"e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc\" (UID: \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\") " pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" Sep 11 12:02:18.203974 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:18.203949 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-workload-certs\") pod \"e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc\" (UID: \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\") " pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" Sep 11 12:02:18.204115 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:18.204002 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-istiod-ca-cert\") pod \"e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc\" (UID: \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\") " pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" Sep 11 12:02:18.205542 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:18.205505 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-istio-podinfo\") pod \"e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc\" (UID: \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\") " pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" Sep 11 12:02:18.205622 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:18.205570 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-istio-envoy\") pod \"e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc\" (UID: \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\") " pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" Sep 11 12:02:18.258397 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:18.258320 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-istio-token\") pod \"e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc\" (UID: \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\") " pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" Sep 11 12:02:18.261740 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:18.261711 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8p4c\" (UniqueName: \"kubernetes.io/projected/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-kube-api-access-j8p4c\") pod \"e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc\" (UID: \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\") " pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" Sep 11 12:02:18.276172 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:18.276149 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" Sep 11 12:02:18.617173 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:18.617138 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc"] Sep 11 12:02:18.621375 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:02:18.621341 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d23b88e_3a6a_472e_ab1e_904ca680e5f4.slice/crio-f24425d26665820580f2cf957a3a5dea93063891ae3ef2a50e82d8c4712f8324 WatchSource:0}: Error finding container f24425d26665820580f2cf957a3a5dea93063891ae3ef2a50e82d8c4712f8324: Status 404 returned error can't find the container with id f24425d26665820580f2cf957a3a5dea93063891ae3ef2a50e82d8c4712f8324 Sep 11 12:02:18.624958 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:18.624914 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:02:18.625227 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:18.625125 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:02:18.625227 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:18.625184 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:02:19.186808 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:19.186772 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" event={"ID":"3d23b88e-3a6a-472e-ab1e-904ca680e5f4","Type":"ContainerStarted","Data":"a68fa1e6673ebb6d21e784f69e7ae1231dcf05ca7a23bff4ca772b8b7efe1aa7"} Sep 11 12:02:19.187194 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:19.186827 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" event={"ID":"3d23b88e-3a6a-472e-ab1e-904ca680e5f4","Type":"ContainerStarted","Data":"f24425d26665820580f2cf957a3a5dea93063891ae3ef2a50e82d8c4712f8324"} Sep 11 12:02:19.277018 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:19.276981 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" Sep 11 12:02:19.282764 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:19.282732 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" Sep 11 12:02:19.324154 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:19.324089 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" podStartSLOduration=2.324066336 podStartE2EDuration="2.324066336s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:02:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:02:19.277940065 +0000 UTC m=+871.042913240" watchObservedRunningTime="2026-09-11 12:02:19.324066336 +0000 UTC m=+871.089039512" Sep 11 12:02:20.191625 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:20.191592 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" Sep 11 12:02:20.192457 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:20.192435 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" Sep 11 12:02:21.177398 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:21.177363 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9" Sep 11 12:02:23.467387 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:23.467349 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-cdec04-9d9dd46d6-4cx82"] Sep 11 12:02:23.470986 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:23.470967 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-cdec04-9d9dd46d6-4cx82" Sep 11 12:02:23.476890 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:23.476846 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-cdec04-dockercfg-zhhtl\"" Sep 11 12:02:23.477441 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:23.477420 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-cdec04\"" Sep 11 12:02:23.485537 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:23.485510 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-cdec04-9d9dd46d6-4cx82"] Sep 11 12:02:23.561381 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:23.561314 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9bef9ee-539a-4bde-b2c3-bfe7d0ccde4e-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-cdec04-9d9dd46d6-4cx82\" (UID: \"f9bef9ee-539a-4bde-b2c3-bfe7d0ccde4e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-cdec04-9d9dd46d6-4cx82" Sep 11 12:02:23.561579 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:23.561440 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf7bf\" (UniqueName: \"kubernetes.io/projected/f9bef9ee-539a-4bde-b2c3-bfe7d0ccde4e-kube-api-access-gf7bf\") pod \"payload-pre-processing-e2e-autores-cdec04-9d9dd46d6-4cx82\" (UID: \"f9bef9ee-539a-4bde-b2c3-bfe7d0ccde4e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-cdec04-9d9dd46d6-4cx82" Sep 11 12:02:23.662687 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:23.662649 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9bef9ee-539a-4bde-b2c3-bfe7d0ccde4e-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-cdec04-9d9dd46d6-4cx82\" (UID: \"f9bef9ee-539a-4bde-b2c3-bfe7d0ccde4e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-cdec04-9d9dd46d6-4cx82" Sep 11 12:02:23.662902 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:23.662710 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf7bf\" (UniqueName: \"kubernetes.io/projected/f9bef9ee-539a-4bde-b2c3-bfe7d0ccde4e-kube-api-access-gf7bf\") pod \"payload-pre-processing-e2e-autores-cdec04-9d9dd46d6-4cx82\" (UID: \"f9bef9ee-539a-4bde-b2c3-bfe7d0ccde4e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-cdec04-9d9dd46d6-4cx82" Sep 11 12:02:23.663391 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:23.663363 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9bef9ee-539a-4bde-b2c3-bfe7d0ccde4e-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-cdec04-9d9dd46d6-4cx82\" (UID: \"f9bef9ee-539a-4bde-b2c3-bfe7d0ccde4e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-cdec04-9d9dd46d6-4cx82" Sep 11 12:02:23.674589 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:23.674559 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf7bf\" (UniqueName: \"kubernetes.io/projected/f9bef9ee-539a-4bde-b2c3-bfe7d0ccde4e-kube-api-access-gf7bf\") pod \"payload-pre-processing-e2e-autores-cdec04-9d9dd46d6-4cx82\" (UID: \"f9bef9ee-539a-4bde-b2c3-bfe7d0ccde4e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-cdec04-9d9dd46d6-4cx82" Sep 11 12:02:23.781111 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:23.781014 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-cdec04-9d9dd46d6-4cx82" Sep 11 12:02:24.102747 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:24.102698 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-cdec04-9d9dd46d6-4cx82"] Sep 11 12:02:24.220846 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:24.220802 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-cdec04-9d9dd46d6-4cx82" event={"ID":"f9bef9ee-539a-4bde-b2c3-bfe7d0ccde4e","Type":"ContainerStarted","Data":"77a1a473cfe47ce7e01a4de3868d8a298b30ab6d342d06ffdeeaa7ad0472c4fc"} Sep 11 12:02:24.221065 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:24.220899 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-cdec04-9d9dd46d6-4cx82" event={"ID":"f9bef9ee-539a-4bde-b2c3-bfe7d0ccde4e","Type":"ContainerStarted","Data":"070d1969fefec3f777c40924df041dbd1b88904b2fb6b7b76e3ac2158c0cbb28"} Sep 11 12:02:24.221264 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:24.221194 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-cdec04-9d9dd46d6-4cx82" Sep 11 12:02:26.174402 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:26.174367 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cbc380-7b4d557d8b-52rrl" Sep 11 12:02:26.197556 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:26.197492 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-cdec04-9d9dd46d6-4cx82" podStartSLOduration=3.19747029 podStartE2EDuration="3.19747029s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:02:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:02:24.326491585 +0000 UTC m=+876.091464761" watchObservedRunningTime="2026-09-11 12:02:26.19747029 +0000 UTC m=+877.962443466" Sep 11 12:02:26.290526 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:26.290494 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-49887c13-79dd6cfd6-rzd96"] Sep 11 12:02:26.293832 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:26.293815 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-49887c13-79dd6cfd6-rzd96" Sep 11 12:02:26.297832 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:26.297803 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-49887c13-dockercfg-cr984\"" Sep 11 12:02:26.298455 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:26.298436 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-49887c13\"" Sep 11 12:02:26.313453 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:26.313423 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-49887c13-79dd6cfd6-rzd96"] Sep 11 12:02:26.359842 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:26.359807 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-49887c13-564db6d8df-98tpc"] Sep 11 12:02:26.363081 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:26.363065 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-49887c13-564db6d8df-98tpc" Sep 11 12:02:26.386726 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:26.386689 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-49887c13-564db6d8df-98tpc"] Sep 11 12:02:26.390329 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:26.390291 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/585a1d07-8b41-4bfe-a6be-afbc69353796-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-49887c13-79dd6cfd6-rzd96\" (UID: \"585a1d07-8b41-4bfe-a6be-afbc69353796\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-49887c13-79dd6cfd6-rzd96" Sep 11 12:02:26.390627 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:26.390601 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed69ef34-c98b-44e5-bd30-ccea47c6a58e-plugins-config-volume\") pod \"payload-processing-e2e-ait-49887c13-564db6d8df-98tpc\" (UID: \"ed69ef34-c98b-44e5-bd30-ccea47c6a58e\") " pod="openshift-ingress/payload-processing-e2e-ait-49887c13-564db6d8df-98tpc" Sep 11 12:02:26.390828 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:26.390809 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p99ks\" (UniqueName: \"kubernetes.io/projected/585a1d07-8b41-4bfe-a6be-afbc69353796-kube-api-access-p99ks\") pod \"payload-pre-processing-e2e-ait-49887c13-79dd6cfd6-rzd96\" (UID: \"585a1d07-8b41-4bfe-a6be-afbc69353796\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-49887c13-79dd6cfd6-rzd96" Sep 11 12:02:26.390977 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:26.390959 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-659hn\" (UniqueName: \"kubernetes.io/projected/ed69ef34-c98b-44e5-bd30-ccea47c6a58e-kube-api-access-659hn\") pod \"payload-processing-e2e-ait-49887c13-564db6d8df-98tpc\" (UID: \"ed69ef34-c98b-44e5-bd30-ccea47c6a58e\") " pod="openshift-ingress/payload-processing-e2e-ait-49887c13-564db6d8df-98tpc" Sep 11 12:02:26.493790 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:26.493695 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed69ef34-c98b-44e5-bd30-ccea47c6a58e-plugins-config-volume\") pod \"payload-processing-e2e-ait-49887c13-564db6d8df-98tpc\" (UID: \"ed69ef34-c98b-44e5-bd30-ccea47c6a58e\") " pod="openshift-ingress/payload-processing-e2e-ait-49887c13-564db6d8df-98tpc" Sep 11 12:02:26.493790 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:26.493760 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p99ks\" (UniqueName: \"kubernetes.io/projected/585a1d07-8b41-4bfe-a6be-afbc69353796-kube-api-access-p99ks\") pod \"payload-pre-processing-e2e-ait-49887c13-79dd6cfd6-rzd96\" (UID: \"585a1d07-8b41-4bfe-a6be-afbc69353796\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-49887c13-79dd6cfd6-rzd96" Sep 11 12:02:26.493790 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:26.493788 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-659hn\" (UniqueName: \"kubernetes.io/projected/ed69ef34-c98b-44e5-bd30-ccea47c6a58e-kube-api-access-659hn\") pod \"payload-processing-e2e-ait-49887c13-564db6d8df-98tpc\" (UID: \"ed69ef34-c98b-44e5-bd30-ccea47c6a58e\") " pod="openshift-ingress/payload-processing-e2e-ait-49887c13-564db6d8df-98tpc" Sep 11 12:02:26.494089 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:26.493909 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/585a1d07-8b41-4bfe-a6be-afbc69353796-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-49887c13-79dd6cfd6-rzd96\" (UID: \"585a1d07-8b41-4bfe-a6be-afbc69353796\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-49887c13-79dd6cfd6-rzd96" Sep 11 12:02:26.494600 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:26.494580 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/585a1d07-8b41-4bfe-a6be-afbc69353796-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-49887c13-79dd6cfd6-rzd96\" (UID: \"585a1d07-8b41-4bfe-a6be-afbc69353796\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-49887c13-79dd6cfd6-rzd96" Sep 11 12:02:26.494639 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:26.494580 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed69ef34-c98b-44e5-bd30-ccea47c6a58e-plugins-config-volume\") pod \"payload-processing-e2e-ait-49887c13-564db6d8df-98tpc\" (UID: \"ed69ef34-c98b-44e5-bd30-ccea47c6a58e\") " pod="openshift-ingress/payload-processing-e2e-ait-49887c13-564db6d8df-98tpc" Sep 11 12:02:26.504979 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:26.504942 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p99ks\" (UniqueName: \"kubernetes.io/projected/585a1d07-8b41-4bfe-a6be-afbc69353796-kube-api-access-p99ks\") pod \"payload-pre-processing-e2e-ait-49887c13-79dd6cfd6-rzd96\" (UID: \"585a1d07-8b41-4bfe-a6be-afbc69353796\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-49887c13-79dd6cfd6-rzd96" Sep 11 12:02:26.505695 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:26.505667 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-659hn\" (UniqueName: \"kubernetes.io/projected/ed69ef34-c98b-44e5-bd30-ccea47c6a58e-kube-api-access-659hn\") pod \"payload-processing-e2e-ait-49887c13-564db6d8df-98tpc\" (UID: \"ed69ef34-c98b-44e5-bd30-ccea47c6a58e\") " pod="openshift-ingress/payload-processing-e2e-ait-49887c13-564db6d8df-98tpc" Sep 11 12:02:26.605045 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:26.605000 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-49887c13-79dd6cfd6-rzd96" Sep 11 12:02:26.675612 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:26.675583 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-49887c13-564db6d8df-98tpc" Sep 11 12:02:26.904761 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:26.904705 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-49887c13-79dd6cfd6-rzd96"] Sep 11 12:02:26.905204 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:02:26.905180 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod585a1d07_8b41_4bfe_a6be_afbc69353796.slice/crio-3326713a2b98ccb59c524361ab7a3e56635bc91259ae6b20e118a2c1468a8814 WatchSource:0}: Error finding container 3326713a2b98ccb59c524361ab7a3e56635bc91259ae6b20e118a2c1468a8814: Status 404 returned error can't find the container with id 3326713a2b98ccb59c524361ab7a3e56635bc91259ae6b20e118a2c1468a8814 Sep 11 12:02:26.988938 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:26.988908 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-49887c13-564db6d8df-98tpc"] Sep 11 12:02:26.991012 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:02:26.990985 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded69ef34_c98b_44e5_bd30_ccea47c6a58e.slice/crio-bbbd90eb29ba6d283965e48b0474b00b99326e34e2c52c84a84458fdb24fce12 WatchSource:0}: Error finding container bbbd90eb29ba6d283965e48b0474b00b99326e34e2c52c84a84458fdb24fce12: Status 404 returned error can't find the container with id bbbd90eb29ba6d283965e48b0474b00b99326e34e2c52c84a84458fdb24fce12 Sep 11 12:02:27.186107 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:27.186009 2566 patch_prober.go:28] interesting pod/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 11 12:02:27.186107 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:27.186087 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" podUID="67bee4aa-84b4-4138-9ecd-51edc10f2e1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 11 12:02:27.242783 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:27.242744 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-49887c13-564db6d8df-98tpc" event={"ID":"ed69ef34-c98b-44e5-bd30-ccea47c6a58e","Type":"ContainerStarted","Data":"490bcebbc1b22c0595f462fc51948e8af88e2a9d8104e64652586e0b0ee3bdfc"} Sep 11 12:02:27.242976 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:27.242807 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-49887c13-564db6d8df-98tpc" event={"ID":"ed69ef34-c98b-44e5-bd30-ccea47c6a58e","Type":"ContainerStarted","Data":"bbbd90eb29ba6d283965e48b0474b00b99326e34e2c52c84a84458fdb24fce12"} Sep 11 12:02:27.242976 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:27.242840 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-49887c13-564db6d8df-98tpc" Sep 11 12:02:27.244576 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:27.244547 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-49887c13-79dd6cfd6-rzd96" event={"ID":"585a1d07-8b41-4bfe-a6be-afbc69353796","Type":"ContainerStarted","Data":"73e271fb4af9db99772fd0deca9bf9e2d4e69f77f241021d8e5b6252a6e99f54"} Sep 11 12:02:27.244713 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:27.244599 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-49887c13-79dd6cfd6-rzd96" event={"ID":"585a1d07-8b41-4bfe-a6be-afbc69353796","Type":"ContainerStarted","Data":"3326713a2b98ccb59c524361ab7a3e56635bc91259ae6b20e118a2c1468a8814"} Sep 11 12:02:27.244713 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:27.244681 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-49887c13-79dd6cfd6-rzd96" Sep 11 12:02:27.392096 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:27.392040 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-49887c13-79dd6cfd6-rzd96" podStartSLOduration=1.3920222899999999 podStartE2EDuration="1.39202229s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:02:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:02:27.390395931 +0000 UTC m=+879.155369107" watchObservedRunningTime="2026-09-11 12:02:27.39202229 +0000 UTC m=+879.156995509" Sep 11 12:02:27.392940 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:27.392900 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-49887c13-564db6d8df-98tpc" podStartSLOduration=1.392885801 podStartE2EDuration="1.392885801s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:02:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:02:27.333724609 +0000 UTC m=+879.098697784" watchObservedRunningTime="2026-09-11 12:02:27.392885801 +0000 UTC m=+879.157858976" Sep 11 12:02:28.251966 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:28.251931 2566 generic.go:356] "Generic (PLEG): container finished" podID="67bee4aa-84b4-4138-9ecd-51edc10f2e1a" containerID="abf3978897df7df56370bfba52738804adc222b5f91c8eaa1d9fb20b24575342" exitCode=0 Sep 11 12:02:28.252487 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:28.252005 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" event={"ID":"67bee4aa-84b4-4138-9ecd-51edc10f2e1a","Type":"ContainerDied","Data":"abf3978897df7df56370bfba52738804adc222b5f91c8eaa1d9fb20b24575342"} Sep 11 12:02:28.252572 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:28.252499 2566 scope.go:117] "RemoveContainer" containerID="abf3978897df7df56370bfba52738804adc222b5f91c8eaa1d9fb20b24575342" Sep 11 12:02:28.253564 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:28.253532 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:02:28.253648 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:28.253610 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:02:28.253648 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:28.253643 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:02:29.259973 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:29.259934 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" event={"ID":"67bee4aa-84b4-4138-9ecd-51edc10f2e1a","Type":"ContainerStarted","Data":"c0c71cb28d33a6c28b1a61a8e64bf7a9d2a8b27974547fa422d7ed34af415c94"} Sep 11 12:02:29.939753 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:29.939703 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" Sep 11 12:02:29.940587 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:29.940556 2566 patch_prober.go:28] interesting pod/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body= Sep 11 12:02:29.940800 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:29.940773 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" podUID="67bee4aa-84b4-4138-9ecd-51edc10f2e1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" Sep 11 12:02:30.940633 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:30.940575 2566 patch_prober.go:28] interesting pod/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body= Sep 11 12:02:30.941221 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:30.940660 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" podUID="67bee4aa-84b4-4138-9ecd-51edc10f2e1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" Sep 11 12:02:31.940465 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:31.940420 2566 patch_prober.go:28] interesting pod/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body= Sep 11 12:02:31.940818 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:31.940777 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" podUID="67bee4aa-84b4-4138-9ecd-51edc10f2e1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" Sep 11 12:02:32.940100 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:32.940061 2566 patch_prober.go:28] interesting pod/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body= Sep 11 12:02:32.940809 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:32.940380 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" podUID="67bee4aa-84b4-4138-9ecd-51edc10f2e1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" Sep 11 12:02:33.940779 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:33.940186 2566 patch_prober.go:28] interesting pod/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body= Sep 11 12:02:33.940779 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:33.940265 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" podUID="67bee4aa-84b4-4138-9ecd-51edc10f2e1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" Sep 11 12:02:34.944641 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:34.944600 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" Sep 11 12:02:34.945108 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:34.945026 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" Sep 11 12:02:34.945636 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:34.945620 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" Sep 11 12:02:35.228311 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:35.228282 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-cdec04-9d9dd46d6-4cx82" Sep 11 12:02:36.234021 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:36.233972 2566 patch_prober.go:28] interesting pod/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 11 12:02:36.234379 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:36.234047 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" podUID="3d23b88e-3a6a-472e-ab1e-904ca680e5f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 11 12:02:38.253971 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:38.253937 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-49887c13-79dd6cfd6-rzd96" Sep 11 12:02:38.254414 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:38.254089 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-49887c13-564db6d8df-98tpc" Sep 11 12:02:38.308895 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:38.308840 2566 generic.go:356] "Generic (PLEG): container finished" podID="3d23b88e-3a6a-472e-ab1e-904ca680e5f4" containerID="a68fa1e6673ebb6d21e784f69e7ae1231dcf05ca7a23bff4ca772b8b7efe1aa7" exitCode=0 Sep 11 12:02:38.309064 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:38.308916 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" event={"ID":"3d23b88e-3a6a-472e-ab1e-904ca680e5f4","Type":"ContainerDied","Data":"a68fa1e6673ebb6d21e784f69e7ae1231dcf05ca7a23bff4ca772b8b7efe1aa7"} Sep 11 12:02:38.309434 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:38.309416 2566 scope.go:117] "RemoveContainer" containerID="a68fa1e6673ebb6d21e784f69e7ae1231dcf05ca7a23bff4ca772b8b7efe1aa7" Sep 11 12:02:38.310719 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:38.310685 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:02:38.310827 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:38.310753 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:02:38.310827 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:38.310786 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:02:39.320126 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:39.320081 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" event={"ID":"3d23b88e-3a6a-472e-ab1e-904ca680e5f4","Type":"ContainerStarted","Data":"b66376aaa1ab8ef03637fb42563e60810461884a18dfce6f5f43589d823b0120"} Sep 11 12:02:40.277159 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:40.277126 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" Sep 11 12:02:40.277394 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:40.277301 2566 patch_prober.go:28] interesting pod/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body= Sep 11 12:02:40.277394 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:40.277347 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" podUID="3d23b88e-3a6a-472e-ab1e-904ca680e5f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" Sep 11 12:02:41.277115 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:41.277025 2566 patch_prober.go:28] interesting pod/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body= Sep 11 12:02:41.277115 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:41.277085 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" podUID="3d23b88e-3a6a-472e-ab1e-904ca680e5f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" Sep 11 12:02:42.276660 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:42.276623 2566 patch_prober.go:28] interesting pod/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body= Sep 11 12:02:42.276837 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:42.276678 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" podUID="3d23b88e-3a6a-472e-ab1e-904ca680e5f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" Sep 11 12:02:43.276468 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:43.276427 2566 patch_prober.go:28] interesting pod/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body= Sep 11 12:02:43.276945 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:43.276494 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" podUID="3d23b88e-3a6a-472e-ab1e-904ca680e5f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" Sep 11 12:02:44.277268 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:44.277230 2566 patch_prober.go:28] interesting pod/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body= Sep 11 12:02:44.277635 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:44.277283 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" podUID="3d23b88e-3a6a-472e-ab1e-904ca680e5f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" Sep 11 12:02:45.280697 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:45.280661 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" Sep 11 12:02:45.281181 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:45.281088 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" Sep 11 12:02:45.281829 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:45.281812 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" Sep 11 12:02:49.321831 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:49.321797 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-12d2a9-5944db5cd9-4wrcl"] Sep 11 12:02:49.324491 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:49.324474 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-12d2a9-5944db5cd9-4wrcl" Sep 11 12:02:49.328590 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:49.328556 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-12d2a9-dockercfg-x9xfk\"" Sep 11 12:02:49.329109 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:49.329089 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-12d2a9\"" Sep 11 12:02:49.356983 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:49.356952 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-12d2a9-5944db5cd9-4wrcl"] Sep 11 12:02:49.419036 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:49.419003 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmphx\" (UniqueName: \"kubernetes.io/projected/efa26d3d-e23b-4e61-a9f6-fe0815ceef93-kube-api-access-rmphx\") pod \"payload-processing-e2e-inf-b-12d2a9-5944db5cd9-4wrcl\" (UID: \"efa26d3d-e23b-4e61-a9f6-fe0815ceef93\") " pod="openshift-ingress/payload-processing-e2e-inf-b-12d2a9-5944db5cd9-4wrcl" Sep 11 12:02:49.419229 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:49.419101 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efa26d3d-e23b-4e61-a9f6-fe0815ceef93-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-12d2a9-5944db5cd9-4wrcl\" (UID: \"efa26d3d-e23b-4e61-a9f6-fe0815ceef93\") " pod="openshift-ingress/payload-processing-e2e-inf-b-12d2a9-5944db5cd9-4wrcl" Sep 11 12:02:49.519675 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:49.519645 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efa26d3d-e23b-4e61-a9f6-fe0815ceef93-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-12d2a9-5944db5cd9-4wrcl\" (UID: \"efa26d3d-e23b-4e61-a9f6-fe0815ceef93\") " pod="openshift-ingress/payload-processing-e2e-inf-b-12d2a9-5944db5cd9-4wrcl" Sep 11 12:02:49.519859 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:49.519713 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmphx\" (UniqueName: \"kubernetes.io/projected/efa26d3d-e23b-4e61-a9f6-fe0815ceef93-kube-api-access-rmphx\") pod \"payload-processing-e2e-inf-b-12d2a9-5944db5cd9-4wrcl\" (UID: \"efa26d3d-e23b-4e61-a9f6-fe0815ceef93\") " pod="openshift-ingress/payload-processing-e2e-inf-b-12d2a9-5944db5cd9-4wrcl" Sep 11 12:02:49.520366 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:49.520343 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efa26d3d-e23b-4e61-a9f6-fe0815ceef93-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-12d2a9-5944db5cd9-4wrcl\" (UID: \"efa26d3d-e23b-4e61-a9f6-fe0815ceef93\") " pod="openshift-ingress/payload-processing-e2e-inf-b-12d2a9-5944db5cd9-4wrcl" Sep 11 12:02:49.544079 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:49.544044 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmphx\" (UniqueName: \"kubernetes.io/projected/efa26d3d-e23b-4e61-a9f6-fe0815ceef93-kube-api-access-rmphx\") pod \"payload-processing-e2e-inf-b-12d2a9-5944db5cd9-4wrcl\" (UID: \"efa26d3d-e23b-4e61-a9f6-fe0815ceef93\") " pod="openshift-ingress/payload-processing-e2e-inf-b-12d2a9-5944db5cd9-4wrcl" Sep 11 12:02:49.634543 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:49.634454 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-12d2a9-5944db5cd9-4wrcl" Sep 11 12:02:49.917901 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:49.917852 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-12d2a9-5944db5cd9-4wrcl"] Sep 11 12:02:49.918688 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:02:49.918664 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefa26d3d_e23b_4e61_a9f6_fe0815ceef93.slice/crio-e1ed3e465ebf9d00725257bfeac6c4dd588dba4bb88ed7bff0fdca01b12c24a2 WatchSource:0}: Error finding container e1ed3e465ebf9d00725257bfeac6c4dd588dba4bb88ed7bff0fdca01b12c24a2: Status 404 returned error can't find the container with id e1ed3e465ebf9d00725257bfeac6c4dd588dba4bb88ed7bff0fdca01b12c24a2 Sep 11 12:02:50.378533 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:50.378494 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-12d2a9-5944db5cd9-4wrcl" event={"ID":"efa26d3d-e23b-4e61-a9f6-fe0815ceef93","Type":"ContainerStarted","Data":"3c4ebd19b0c03f7c86c9ff429547c880df6d74c13d392c0255abcd77b4413dfa"} Sep 11 12:02:50.378989 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:50.378558 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-12d2a9-5944db5cd9-4wrcl" event={"ID":"efa26d3d-e23b-4e61-a9f6-fe0815ceef93","Type":"ContainerStarted","Data":"e1ed3e465ebf9d00725257bfeac6c4dd588dba4bb88ed7bff0fdca01b12c24a2"} Sep 11 12:02:50.378989 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:50.378612 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-12d2a9-5944db5cd9-4wrcl" Sep 11 12:02:50.399431 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:02:50.399381 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-12d2a9-5944db5cd9-4wrcl" podStartSLOduration=1.399362674 podStartE2EDuration="1.399362674s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:02:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:02:50.399214365 +0000 UTC m=+902.164187540" watchObservedRunningTime="2026-09-11 12:02:50.399362674 +0000 UTC m=+902.164335851" Sep 11 12:03:01.391354 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:01.391324 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-12d2a9-5944db5cd9-4wrcl" Sep 11 12:03:06.297543 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:06.297508 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-49887c13-564db6d8df-98tpc"] Sep 11 12:03:06.298018 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:06.297782 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-49887c13-564db6d8df-98tpc" podUID="ed69ef34-c98b-44e5-bd30-ccea47c6a58e" containerName="payload-processing" containerID="cri-o://490bcebbc1b22c0595f462fc51948e8af88e2a9d8104e64652586e0b0ee3bdfc" gracePeriod=30 Sep 11 12:03:06.313663 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:06.313586 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-49887c13-79dd6cfd6-rzd96"] Sep 11 12:03:06.313880 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:06.313801 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-49887c13-79dd6cfd6-rzd96" podUID="585a1d07-8b41-4bfe-a6be-afbc69353796" containerName="payload-pre-processing" containerID="cri-o://73e271fb4af9db99772fd0deca9bf9e2d4e69f77f241021d8e5b6252a6e99f54" gracePeriod=30 Sep 11 12:03:06.465320 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:06.465280 2566 generic.go:356] "Generic (PLEG): container finished" podID="585a1d07-8b41-4bfe-a6be-afbc69353796" containerID="73e271fb4af9db99772fd0deca9bf9e2d4e69f77f241021d8e5b6252a6e99f54" exitCode=0 Sep 11 12:03:06.465502 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:06.465359 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-49887c13-79dd6cfd6-rzd96" event={"ID":"585a1d07-8b41-4bfe-a6be-afbc69353796","Type":"ContainerDied","Data":"73e271fb4af9db99772fd0deca9bf9e2d4e69f77f241021d8e5b6252a6e99f54"} Sep 11 12:03:06.467512 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:06.467484 2566 generic.go:356] "Generic (PLEG): container finished" podID="ed69ef34-c98b-44e5-bd30-ccea47c6a58e" containerID="490bcebbc1b22c0595f462fc51948e8af88e2a9d8104e64652586e0b0ee3bdfc" exitCode=0 Sep 11 12:03:06.467649 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:06.467559 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-49887c13-564db6d8df-98tpc" event={"ID":"ed69ef34-c98b-44e5-bd30-ccea47c6a58e","Type":"ContainerDied","Data":"490bcebbc1b22c0595f462fc51948e8af88e2a9d8104e64652586e0b0ee3bdfc"} Sep 11 12:03:06.568583 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:06.568560 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-49887c13-564db6d8df-98tpc" Sep 11 12:03:06.591754 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:06.591730 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-49887c13-79dd6cfd6-rzd96" Sep 11 12:03:06.658438 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:06.658376 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-659hn\" (UniqueName: \"kubernetes.io/projected/ed69ef34-c98b-44e5-bd30-ccea47c6a58e-kube-api-access-659hn\") pod \"ed69ef34-c98b-44e5-bd30-ccea47c6a58e\" (UID: \"ed69ef34-c98b-44e5-bd30-ccea47c6a58e\") " Sep 11 12:03:06.658605 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:06.658473 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/585a1d07-8b41-4bfe-a6be-afbc69353796-plugins-config-volume\") pod \"585a1d07-8b41-4bfe-a6be-afbc69353796\" (UID: \"585a1d07-8b41-4bfe-a6be-afbc69353796\") " Sep 11 12:03:06.658694 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:06.658649 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed69ef34-c98b-44e5-bd30-ccea47c6a58e-plugins-config-volume\") pod \"ed69ef34-c98b-44e5-bd30-ccea47c6a58e\" (UID: \"ed69ef34-c98b-44e5-bd30-ccea47c6a58e\") " Sep 11 12:03:06.658753 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:06.658740 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p99ks\" (UniqueName: \"kubernetes.io/projected/585a1d07-8b41-4bfe-a6be-afbc69353796-kube-api-access-p99ks\") pod \"585a1d07-8b41-4bfe-a6be-afbc69353796\" (UID: \"585a1d07-8b41-4bfe-a6be-afbc69353796\") " Sep 11 12:03:06.659189 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:06.659140 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed69ef34-c98b-44e5-bd30-ccea47c6a58e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ed69ef34-c98b-44e5-bd30-ccea47c6a58e" (UID: "ed69ef34-c98b-44e5-bd30-ccea47c6a58e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 12:03:06.659189 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:06.659154 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/585a1d07-8b41-4bfe-a6be-afbc69353796-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "585a1d07-8b41-4bfe-a6be-afbc69353796" (UID: "585a1d07-8b41-4bfe-a6be-afbc69353796"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 12:03:06.663200 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:06.663162 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/585a1d07-8b41-4bfe-a6be-afbc69353796-kube-api-access-p99ks" (OuterVolumeSpecName: "kube-api-access-p99ks") pod "585a1d07-8b41-4bfe-a6be-afbc69353796" (UID: "585a1d07-8b41-4bfe-a6be-afbc69353796"). InnerVolumeSpecName "kube-api-access-p99ks". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:03:06.663343 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:06.663210 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed69ef34-c98b-44e5-bd30-ccea47c6a58e-kube-api-access-659hn" (OuterVolumeSpecName: "kube-api-access-659hn") pod "ed69ef34-c98b-44e5-bd30-ccea47c6a58e" (UID: "ed69ef34-c98b-44e5-bd30-ccea47c6a58e"). InnerVolumeSpecName "kube-api-access-659hn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:03:06.759754 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:06.759718 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p99ks\" (UniqueName: \"kubernetes.io/projected/585a1d07-8b41-4bfe-a6be-afbc69353796-kube-api-access-p99ks\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:03:06.759754 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:06.759753 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-659hn\" (UniqueName: \"kubernetes.io/projected/ed69ef34-c98b-44e5-bd30-ccea47c6a58e-kube-api-access-659hn\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:03:06.760008 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:06.759768 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/585a1d07-8b41-4bfe-a6be-afbc69353796-plugins-config-volume\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:03:06.760008 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:06.759781 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed69ef34-c98b-44e5-bd30-ccea47c6a58e-plugins-config-volume\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:03:07.473484 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:07.473453 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-49887c13-564db6d8df-98tpc" Sep 11 12:03:07.473917 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:07.473489 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-49887c13-564db6d8df-98tpc" event={"ID":"ed69ef34-c98b-44e5-bd30-ccea47c6a58e","Type":"ContainerDied","Data":"bbbd90eb29ba6d283965e48b0474b00b99326e34e2c52c84a84458fdb24fce12"} Sep 11 12:03:07.473917 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:07.473542 2566 scope.go:117] "RemoveContainer" containerID="490bcebbc1b22c0595f462fc51948e8af88e2a9d8104e64652586e0b0ee3bdfc" Sep 11 12:03:07.474903 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:07.474886 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-49887c13-79dd6cfd6-rzd96" event={"ID":"585a1d07-8b41-4bfe-a6be-afbc69353796","Type":"ContainerDied","Data":"3326713a2b98ccb59c524361ab7a3e56635bc91259ae6b20e118a2c1468a8814"} Sep 11 12:03:07.474971 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:07.474925 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-49887c13-79dd6cfd6-rzd96" Sep 11 12:03:07.483780 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:07.483763 2566 scope.go:117] "RemoveContainer" containerID="73e271fb4af9db99772fd0deca9bf9e2d4e69f77f241021d8e5b6252a6e99f54" Sep 11 12:03:07.514622 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:07.514597 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-49887c13-564db6d8df-98tpc"] Sep 11 12:03:07.534328 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:07.534303 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-49887c13-564db6d8df-98tpc"] Sep 11 12:03:07.559288 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:07.559249 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-49887c13-79dd6cfd6-rzd96"] Sep 11 12:03:07.566760 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:07.566737 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-49887c13-79dd6cfd6-rzd96"] Sep 11 12:03:08.758521 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:08.758477 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="585a1d07-8b41-4bfe-a6be-afbc69353796" path="/var/lib/kubelet/pods/585a1d07-8b41-4bfe-a6be-afbc69353796/volumes" Sep 11 12:03:08.759045 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:08.759026 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed69ef34-c98b-44e5-bd30-ccea47c6a58e" path="/var/lib/kubelet/pods/ed69ef34-c98b-44e5-bd30-ccea47c6a58e/volumes" Sep 11 12:03:43.502025 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:43.501981 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd"] Sep 11 12:03:43.502445 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:43.502330 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="585a1d07-8b41-4bfe-a6be-afbc69353796" containerName="payload-pre-processing" Sep 11 12:03:43.502445 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:43.502343 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="585a1d07-8b41-4bfe-a6be-afbc69353796" containerName="payload-pre-processing" Sep 11 12:03:43.502445 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:43.502350 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed69ef34-c98b-44e5-bd30-ccea47c6a58e" containerName="payload-processing" Sep 11 12:03:43.502445 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:43.502358 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed69ef34-c98b-44e5-bd30-ccea47c6a58e" containerName="payload-processing" Sep 11 12:03:43.502445 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:43.502431 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="585a1d07-8b41-4bfe-a6be-afbc69353796" containerName="payload-pre-processing" Sep 11 12:03:43.502445 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:43.502438 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed69ef34-c98b-44e5-bd30-ccea47c6a58e" containerName="payload-processing" Sep 11 12:03:43.504696 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:43.504675 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" Sep 11 12:03:43.508107 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:43.508086 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-087015a2-openshift-default-dockercfg-99dbh\"" Sep 11 12:03:43.510029 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:43.510004 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-istio-data\") pod \"e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd\" (UID: \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\") " pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" Sep 11 12:03:43.510128 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:43.510046 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-istio-envoy\") pod \"e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd\" (UID: \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\") " pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" Sep 11 12:03:43.510128 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:43.510071 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8szhf\" (UniqueName: \"kubernetes.io/projected/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-kube-api-access-8szhf\") pod \"e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd\" (UID: \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\") " pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" Sep 11 12:03:43.510128 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:43.510122 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-istiod-ca-cert\") pod \"e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd\" (UID: \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\") " pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" Sep 11 12:03:43.510280 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:43.510178 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-workload-socket\") pod \"e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd\" (UID: \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\") " pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" Sep 11 12:03:43.510280 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:43.510206 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-istio-podinfo\") pod \"e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd\" (UID: \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\") " pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" Sep 11 12:03:43.510375 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:43.510300 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-credential-socket\") pod \"e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd\" (UID: \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\") " pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" Sep 11 12:03:43.510375 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:43.510335 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-istio-token\") pod \"e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd\" (UID: \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\") " pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" Sep 11 12:03:43.510375 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:43.510371 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-workload-certs\") pod \"e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd\" (UID: \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\") " pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" Sep 11 12:03:43.523639 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:43.523616 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd"] Sep 11 12:03:43.611760 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:43.611722 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-workload-socket\") pod \"e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd\" (UID: \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\") " pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" Sep 11 12:03:43.612005 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:43.611774 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-istio-podinfo\") pod \"e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd\" (UID: \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\") " pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" Sep 11 12:03:43.612005 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:43.611841 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-credential-socket\") pod \"e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd\" (UID: \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\") " pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" Sep 11 12:03:43.612005 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:43.611888 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-istio-token\") pod \"e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd\" (UID: \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\") " pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" Sep 11 12:03:43.612005 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:43.611928 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-workload-certs\") pod \"e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd\" (UID: \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\") " pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" Sep 11 12:03:43.612005 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:43.611972 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-istio-data\") pod \"e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd\" (UID: \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\") " pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" Sep 11 12:03:43.612005 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:43.611998 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-istio-envoy\") pod \"e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd\" (UID: \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\") " pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" Sep 11 12:03:43.612330 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:43.612020 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8szhf\" (UniqueName: \"kubernetes.io/projected/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-kube-api-access-8szhf\") pod \"e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd\" (UID: \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\") " pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" Sep 11 12:03:43.612330 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:43.612069 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-istiod-ca-cert\") pod \"e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd\" (UID: \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\") " pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" Sep 11 12:03:43.612905 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:43.612861 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-istiod-ca-cert\") pod \"e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd\" (UID: \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\") " pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" Sep 11 12:03:43.613699 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:43.613582 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-credential-socket\") pod \"e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd\" (UID: \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\") " pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" Sep 11 12:03:43.614112 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:43.613850 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-workload-certs\") pod \"e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd\" (UID: \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\") " pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" Sep 11 12:03:43.614255 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:43.613878 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-istio-data\") pod \"e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd\" (UID: \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\") " pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" Sep 11 12:03:43.614361 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:43.614134 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-workload-socket\") pod \"e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd\" (UID: \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\") " pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" Sep 11 12:03:43.616655 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:43.616635 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-istio-envoy\") pod \"e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd\" (UID: \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\") " pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" Sep 11 12:03:43.616971 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:43.616794 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-istio-podinfo\") pod \"e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd\" (UID: \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\") " pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" Sep 11 12:03:43.629292 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:43.629262 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-istio-token\") pod \"e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd\" (UID: \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\") " pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" Sep 11 12:03:43.629448 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:43.629429 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8szhf\" (UniqueName: \"kubernetes.io/projected/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-kube-api-access-8szhf\") pod \"e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd\" (UID: \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\") " pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" Sep 11 12:03:43.814622 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:43.814540 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" Sep 11 12:03:44.107392 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:44.107363 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd"] Sep 11 12:03:44.109716 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:03:44.109681 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0b95a4e_4b47_48da_bb25_a9e7b9b0de1a.slice/crio-670dc5a2b98098f57d11e9642a0a6fe29e36b07d35bc7606c037303adddd1172 WatchSource:0}: Error finding container 670dc5a2b98098f57d11e9642a0a6fe29e36b07d35bc7606c037303adddd1172: Status 404 returned error can't find the container with id 670dc5a2b98098f57d11e9642a0a6fe29e36b07d35bc7606c037303adddd1172 Sep 11 12:03:44.111998 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:44.111963 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:03:44.112071 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:44.112034 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:03:44.112071 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:44.112061 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:03:44.651268 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:44.651228 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" event={"ID":"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a","Type":"ContainerStarted","Data":"1ef91bff6d52134c0d2f2d2c968d9f955cbf59733359a317140f6ab8f6841d56"} Sep 11 12:03:44.651639 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:44.651285 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" event={"ID":"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a","Type":"ContainerStarted","Data":"670dc5a2b98098f57d11e9642a0a6fe29e36b07d35bc7606c037303adddd1172"} Sep 11 12:03:44.679051 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:44.678994 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" podStartSLOduration=1.678978927 podStartE2EDuration="1.678978927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:03:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:03:44.676573438 +0000 UTC m=+956.441546614" watchObservedRunningTime="2026-09-11 12:03:44.678978927 +0000 UTC m=+956.443952102" Sep 11 12:03:44.815269 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:44.815235 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" Sep 11 12:03:45.820971 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:45.820940 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" Sep 11 12:03:46.661737 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:46.661705 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" Sep 11 12:03:46.662590 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:03:46.662565 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" Sep 11 12:04:07.774140 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:07.774106 2566 generic.go:356] "Generic (PLEG): container finished" podID="b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a" containerID="1ef91bff6d52134c0d2f2d2c968d9f955cbf59733359a317140f6ab8f6841d56" exitCode=0 Sep 11 12:04:07.774552 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:07.774156 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" event={"ID":"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a","Type":"ContainerDied","Data":"1ef91bff6d52134c0d2f2d2c968d9f955cbf59733359a317140f6ab8f6841d56"} Sep 11 12:04:07.774552 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:07.774515 2566 scope.go:117] "RemoveContainer" containerID="1ef91bff6d52134c0d2f2d2c968d9f955cbf59733359a317140f6ab8f6841d56" Sep 11 12:04:07.775577 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:07.775545 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:04:07.775675 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:07.775660 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:04:07.775725 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:07.775698 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:04:08.780194 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:08.780157 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" event={"ID":"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a","Type":"ContainerStarted","Data":"29ea1096f318b7052d3e3df0c0e2d7734d32e4235059660d04ff672e8c52dfc8"} Sep 11 12:04:08.815453 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:08.815417 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" Sep 11 12:04:08.815619 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:08.815581 2566 patch_prober.go:28] interesting pod/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Sep 11 12:04:08.815668 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:08.815623 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" podUID="b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Sep 11 12:04:09.815785 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:09.815742 2566 patch_prober.go:28] interesting pod/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Sep 11 12:04:09.816426 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:09.816389 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" podUID="b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Sep 11 12:04:10.816073 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:10.816031 2566 patch_prober.go:28] interesting pod/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Sep 11 12:04:10.816558 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:10.816102 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" podUID="b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Sep 11 12:04:11.815163 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:11.815116 2566 patch_prober.go:28] interesting pod/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Sep 11 12:04:11.815369 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:11.815234 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" podUID="b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Sep 11 12:04:12.815638 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:12.815590 2566 patch_prober.go:28] interesting pod/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Sep 11 12:04:12.816144 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:12.815662 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" podUID="b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Sep 11 12:04:13.815339 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:13.815292 2566 patch_prober.go:28] interesting pod/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Sep 11 12:04:13.815555 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:13.815364 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" podUID="b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Sep 11 12:04:14.820042 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:14.820004 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" Sep 11 12:04:14.820501 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:14.820386 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" Sep 11 12:04:14.826131 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:14.821641 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" Sep 11 12:04:20.528168 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:20.528126 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd"] Sep 11 12:04:20.528743 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:20.528407 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" podUID="b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a" containerName="istio-proxy" containerID="cri-o://29ea1096f318b7052d3e3df0c0e2d7734d32e4235059660d04ff672e8c52dfc8" gracePeriod=30 Sep 11 12:04:21.274691 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:21.274645 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-cdec04-9d9dd46d6-4cx82"] Sep 11 12:04:21.275268 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:21.275233 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-cdec04-9d9dd46d6-4cx82" podUID="f9bef9ee-539a-4bde-b2c3-bfe7d0ccde4e" containerName="payload-pre-processing" containerID="cri-o://77a1a473cfe47ce7e01a4de3868d8a298b30ab6d342d06ffdeeaa7ad0472c4fc" gracePeriod=30 Sep 11 12:04:21.531959 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:21.531839 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-cdec04-9d9dd46d6-4cx82" Sep 11 12:04:21.687535 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:21.687500 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9bef9ee-539a-4bde-b2c3-bfe7d0ccde4e-plugins-config-volume\") pod \"f9bef9ee-539a-4bde-b2c3-bfe7d0ccde4e\" (UID: \"f9bef9ee-539a-4bde-b2c3-bfe7d0ccde4e\") " Sep 11 12:04:21.687702 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:21.687608 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gf7bf\" (UniqueName: \"kubernetes.io/projected/f9bef9ee-539a-4bde-b2c3-bfe7d0ccde4e-kube-api-access-gf7bf\") pod \"f9bef9ee-539a-4bde-b2c3-bfe7d0ccde4e\" (UID: \"f9bef9ee-539a-4bde-b2c3-bfe7d0ccde4e\") " Sep 11 12:04:21.687915 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:21.687890 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9bef9ee-539a-4bde-b2c3-bfe7d0ccde4e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f9bef9ee-539a-4bde-b2c3-bfe7d0ccde4e" (UID: "f9bef9ee-539a-4bde-b2c3-bfe7d0ccde4e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 12:04:21.689630 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:21.689600 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9bef9ee-539a-4bde-b2c3-bfe7d0ccde4e-kube-api-access-gf7bf" (OuterVolumeSpecName: "kube-api-access-gf7bf") pod "f9bef9ee-539a-4bde-b2c3-bfe7d0ccde4e" (UID: "f9bef9ee-539a-4bde-b2c3-bfe7d0ccde4e"). InnerVolumeSpecName "kube-api-access-gf7bf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:04:21.788458 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:21.788377 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gf7bf\" (UniqueName: \"kubernetes.io/projected/f9bef9ee-539a-4bde-b2c3-bfe7d0ccde4e-kube-api-access-gf7bf\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:04:21.788458 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:21.788405 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9bef9ee-539a-4bde-b2c3-bfe7d0ccde4e-plugins-config-volume\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:04:21.845592 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:21.845556 2566 generic.go:356] "Generic (PLEG): container finished" podID="f9bef9ee-539a-4bde-b2c3-bfe7d0ccde4e" containerID="77a1a473cfe47ce7e01a4de3868d8a298b30ab6d342d06ffdeeaa7ad0472c4fc" exitCode=0 Sep 11 12:04:21.845772 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:21.845606 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-cdec04-9d9dd46d6-4cx82" Sep 11 12:04:21.845772 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:21.845640 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-cdec04-9d9dd46d6-4cx82" event={"ID":"f9bef9ee-539a-4bde-b2c3-bfe7d0ccde4e","Type":"ContainerDied","Data":"77a1a473cfe47ce7e01a4de3868d8a298b30ab6d342d06ffdeeaa7ad0472c4fc"} Sep 11 12:04:21.845772 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:21.845700 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-cdec04-9d9dd46d6-4cx82" event={"ID":"f9bef9ee-539a-4bde-b2c3-bfe7d0ccde4e","Type":"ContainerDied","Data":"070d1969fefec3f777c40924df041dbd1b88904b2fb6b7b76e3ac2158c0cbb28"} Sep 11 12:04:21.845772 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:21.845724 2566 scope.go:117] "RemoveContainer" containerID="77a1a473cfe47ce7e01a4de3868d8a298b30ab6d342d06ffdeeaa7ad0472c4fc" Sep 11 12:04:21.854766 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:21.854744 2566 scope.go:117] "RemoveContainer" containerID="77a1a473cfe47ce7e01a4de3868d8a298b30ab6d342d06ffdeeaa7ad0472c4fc" Sep 11 12:04:21.855057 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:04:21.855040 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77a1a473cfe47ce7e01a4de3868d8a298b30ab6d342d06ffdeeaa7ad0472c4fc\": container with ID starting with 77a1a473cfe47ce7e01a4de3868d8a298b30ab6d342d06ffdeeaa7ad0472c4fc not found: ID does not exist" containerID="77a1a473cfe47ce7e01a4de3868d8a298b30ab6d342d06ffdeeaa7ad0472c4fc" Sep 11 12:04:21.855120 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:21.855068 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77a1a473cfe47ce7e01a4de3868d8a298b30ab6d342d06ffdeeaa7ad0472c4fc"} err="failed to get container status \"77a1a473cfe47ce7e01a4de3868d8a298b30ab6d342d06ffdeeaa7ad0472c4fc\": rpc error: code = NotFound desc = could not find container \"77a1a473cfe47ce7e01a4de3868d8a298b30ab6d342d06ffdeeaa7ad0472c4fc\": container with ID starting with 77a1a473cfe47ce7e01a4de3868d8a298b30ab6d342d06ffdeeaa7ad0472c4fc not found: ID does not exist" Sep 11 12:04:21.874230 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:21.874205 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-cdec04-9d9dd46d6-4cx82"] Sep 11 12:04:21.879446 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:21.879424 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-cdec04-9d9dd46d6-4cx82"] Sep 11 12:04:22.758136 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:22.758101 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9bef9ee-539a-4bde-b2c3-bfe7d0ccde4e" path="/var/lib/kubelet/pods/f9bef9ee-539a-4bde-b2c3-bfe7d0ccde4e/volumes" Sep 11 12:04:25.540551 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:04:25.540427 2566 helpers.go:245] readString: Failed to read "/sys/fs/cgroup/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0b95a4e_4b47_48da_bb25_a9e7b9b0de1a.slice/crio-29ea1096f318b7052d3e3df0c0e2d7734d32e4235059660d04ff672e8c52dfc8.scope/pids.max": read /sys/fs/cgroup/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0b95a4e_4b47_48da_bb25_a9e7b9b0de1a.slice/crio-29ea1096f318b7052d3e3df0c0e2d7734d32e4235059660d04ff672e8c52dfc8.scope/pids.max: no such device Sep 11 12:04:25.788683 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:25.788655 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" Sep 11 12:04:25.867075 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:25.867038 2566 generic.go:356] "Generic (PLEG): container finished" podID="b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a" containerID="29ea1096f318b7052d3e3df0c0e2d7734d32e4235059660d04ff672e8c52dfc8" exitCode=0 Sep 11 12:04:25.867281 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:25.867092 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" Sep 11 12:04:25.867281 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:25.867129 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" event={"ID":"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a","Type":"ContainerDied","Data":"29ea1096f318b7052d3e3df0c0e2d7734d32e4235059660d04ff672e8c52dfc8"} Sep 11 12:04:25.867281 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:25.867193 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd" event={"ID":"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a","Type":"ContainerDied","Data":"670dc5a2b98098f57d11e9642a0a6fe29e36b07d35bc7606c037303adddd1172"} Sep 11 12:04:25.867281 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:25.867226 2566 scope.go:117] "RemoveContainer" containerID="29ea1096f318b7052d3e3df0c0e2d7734d32e4235059660d04ff672e8c52dfc8" Sep 11 12:04:25.875548 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:25.875529 2566 scope.go:117] "RemoveContainer" containerID="1ef91bff6d52134c0d2f2d2c968d9f955cbf59733359a317140f6ab8f6841d56" Sep 11 12:04:25.882960 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:25.882943 2566 scope.go:117] "RemoveContainer" containerID="29ea1096f318b7052d3e3df0c0e2d7734d32e4235059660d04ff672e8c52dfc8" Sep 11 12:04:25.883221 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:04:25.883201 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29ea1096f318b7052d3e3df0c0e2d7734d32e4235059660d04ff672e8c52dfc8\": container with ID starting with 29ea1096f318b7052d3e3df0c0e2d7734d32e4235059660d04ff672e8c52dfc8 not found: ID does not exist" containerID="29ea1096f318b7052d3e3df0c0e2d7734d32e4235059660d04ff672e8c52dfc8" Sep 11 12:04:25.883277 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:25.883230 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29ea1096f318b7052d3e3df0c0e2d7734d32e4235059660d04ff672e8c52dfc8"} err="failed to get container status \"29ea1096f318b7052d3e3df0c0e2d7734d32e4235059660d04ff672e8c52dfc8\": rpc error: code = NotFound desc = could not find container \"29ea1096f318b7052d3e3df0c0e2d7734d32e4235059660d04ff672e8c52dfc8\": container with ID starting with 29ea1096f318b7052d3e3df0c0e2d7734d32e4235059660d04ff672e8c52dfc8 not found: ID does not exist" Sep 11 12:04:25.883277 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:25.883249 2566 scope.go:117] "RemoveContainer" containerID="1ef91bff6d52134c0d2f2d2c968d9f955cbf59733359a317140f6ab8f6841d56" Sep 11 12:04:25.883506 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:04:25.883485 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ef91bff6d52134c0d2f2d2c968d9f955cbf59733359a317140f6ab8f6841d56\": container with ID starting with 1ef91bff6d52134c0d2f2d2c968d9f955cbf59733359a317140f6ab8f6841d56 not found: ID does not exist" containerID="1ef91bff6d52134c0d2f2d2c968d9f955cbf59733359a317140f6ab8f6841d56" Sep 11 12:04:25.883546 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:25.883512 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ef91bff6d52134c0d2f2d2c968d9f955cbf59733359a317140f6ab8f6841d56"} err="failed to get container status \"1ef91bff6d52134c0d2f2d2c968d9f955cbf59733359a317140f6ab8f6841d56\": rpc error: code = NotFound desc = could not find container \"1ef91bff6d52134c0d2f2d2c968d9f955cbf59733359a317140f6ab8f6841d56\": container with ID starting with 1ef91bff6d52134c0d2f2d2c968d9f955cbf59733359a317140f6ab8f6841d56 not found: ID does not exist" Sep 11 12:04:25.927627 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:25.927592 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-istio-data\") pod \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\" (UID: \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\") " Sep 11 12:04:25.927765 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:25.927639 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-istio-token\") pod \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\" (UID: \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\") " Sep 11 12:04:25.927765 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:25.927665 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-workload-certs\") pod \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\" (UID: \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\") " Sep 11 12:04:25.927765 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:25.927691 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-credential-socket\") pod \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\" (UID: \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\") " Sep 11 12:04:25.927765 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:25.927710 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8szhf\" (UniqueName: \"kubernetes.io/projected/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-kube-api-access-8szhf\") pod \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\" (UID: \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\") " Sep 11 12:04:25.927765 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:25.927763 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-istio-envoy\") pod \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\" (UID: \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\") " Sep 11 12:04:25.928049 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:25.927786 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-istiod-ca-cert\") pod \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\" (UID: \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\") " Sep 11 12:04:25.928049 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:25.927820 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-istio-podinfo\") pod \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\" (UID: \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\") " Sep 11 12:04:25.928049 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:25.927906 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-workload-socket\") pod \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\" (UID: \"b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a\") " Sep 11 12:04:25.928209 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:25.927897 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-istio-data" (OuterVolumeSpecName: "istio-data") pod "b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a" (UID: "b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:04:25.928294 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:25.928063 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a" (UID: "b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:04:25.928348 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:25.928153 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a" (UID: "b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 12:04:25.928348 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:25.928248 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a" (UID: "b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:04:25.928348 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:25.928261 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a" (UID: "b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:04:25.928348 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:25.928337 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-istio-data\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:04:25.930063 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:25.930035 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-istio-token" (OuterVolumeSpecName: "istio-token") pod "b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a" (UID: "b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:04:25.930172 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:25.930078 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a" (UID: "b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 11 12:04:25.930172 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:25.930075 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-kube-api-access-8szhf" (OuterVolumeSpecName: "kube-api-access-8szhf") pod "b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a" (UID: "b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a"). InnerVolumeSpecName "kube-api-access-8szhf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:04:25.930321 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:25.930296 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a" (UID: "b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:04:26.029812 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:26.029780 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-istio-token\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:04:26.029812 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:26.029813 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-workload-certs\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:04:26.030022 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:26.029824 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-credential-socket\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:04:26.030022 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:26.029834 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8szhf\" (UniqueName: \"kubernetes.io/projected/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-kube-api-access-8szhf\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:04:26.030022 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:26.029843 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-istio-envoy\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:04:26.030022 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:26.029851 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-istiod-ca-cert\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:04:26.030022 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:26.029859 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-istio-podinfo\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:04:26.030022 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:26.029892 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a-workload-socket\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:04:26.191933 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:26.191905 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd"] Sep 11 12:04:26.195551 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:26.195526 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-087015a2-openshift-default-76dfdf794d-xttfd"] Sep 11 12:04:26.758025 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:04:26.757988 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a" path="/var/lib/kubelet/pods/b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a/volumes" Sep 11 12:05:05.145041 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:05.145002 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc"] Sep 11 12:05:05.145536 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:05.145279 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" podUID="3d23b88e-3a6a-472e-ab1e-904ca680e5f4" containerName="istio-proxy" containerID="cri-o://b66376aaa1ab8ef03637fb42563e60810461884a18dfce6f5f43589d823b0120" gracePeriod=30 Sep 11 12:05:05.837042 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:05.837009 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89"] Sep 11 12:05:05.837365 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:05.837353 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9bef9ee-539a-4bde-b2c3-bfe7d0ccde4e" containerName="payload-pre-processing" Sep 11 12:05:05.837408 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:05.837367 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9bef9ee-539a-4bde-b2c3-bfe7d0ccde4e" containerName="payload-pre-processing" Sep 11 12:05:05.837408 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:05.837378 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a" containerName="istio-proxy" Sep 11 12:05:05.837408 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:05.837383 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a" containerName="istio-proxy" Sep 11 12:05:05.837500 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:05.837447 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a" containerName="istio-proxy" Sep 11 12:05:05.837500 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:05.837455 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9bef9ee-539a-4bde-b2c3-bfe7d0ccde4e" containerName="payload-pre-processing" Sep 11 12:05:05.837500 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:05.837461 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a" containerName="istio-proxy" Sep 11 12:05:05.840222 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:05.840205 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" Sep 11 12:05:05.843259 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:05.843239 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-69b1ea-openshift-default-dockercfg-ggw8q\"" Sep 11 12:05:05.855443 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:05.855416 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89"] Sep 11 12:05:05.985174 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:05.985141 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcf12e46-656a-4d89-962a-b1a6cefaf643-istio-podinfo\") pod \"e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89\" (UID: \"bcf12e46-656a-4d89-962a-b1a6cefaf643\") " pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" Sep 11 12:05:05.985328 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:05.985192 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6fs8\" (UniqueName: \"kubernetes.io/projected/bcf12e46-656a-4d89-962a-b1a6cefaf643-kube-api-access-x6fs8\") pod \"e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89\" (UID: \"bcf12e46-656a-4d89-962a-b1a6cefaf643\") " pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" Sep 11 12:05:05.985328 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:05.985240 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcf12e46-656a-4d89-962a-b1a6cefaf643-istiod-ca-cert\") pod \"e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89\" (UID: \"bcf12e46-656a-4d89-962a-b1a6cefaf643\") " pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" Sep 11 12:05:05.985328 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:05.985292 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcf12e46-656a-4d89-962a-b1a6cefaf643-istio-token\") pod \"e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89\" (UID: \"bcf12e46-656a-4d89-962a-b1a6cefaf643\") " pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" Sep 11 12:05:05.985328 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:05.985326 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcf12e46-656a-4d89-962a-b1a6cefaf643-istio-envoy\") pod \"e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89\" (UID: \"bcf12e46-656a-4d89-962a-b1a6cefaf643\") " pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" Sep 11 12:05:05.985480 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:05.985364 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf12e46-656a-4d89-962a-b1a6cefaf643-workload-socket\") pod \"e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89\" (UID: \"bcf12e46-656a-4d89-962a-b1a6cefaf643\") " pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" Sep 11 12:05:05.985480 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:05.985432 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcf12e46-656a-4d89-962a-b1a6cefaf643-istio-data\") pod \"e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89\" (UID: \"bcf12e46-656a-4d89-962a-b1a6cefaf643\") " pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" Sep 11 12:05:05.985480 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:05.985465 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf12e46-656a-4d89-962a-b1a6cefaf643-credential-socket\") pod \"e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89\" (UID: \"bcf12e46-656a-4d89-962a-b1a6cefaf643\") " pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" Sep 11 12:05:05.985603 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:05.985481 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcf12e46-656a-4d89-962a-b1a6cefaf643-workload-certs\") pod \"e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89\" (UID: \"bcf12e46-656a-4d89-962a-b1a6cefaf643\") " pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" Sep 11 12:05:06.086014 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:06.085974 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcf12e46-656a-4d89-962a-b1a6cefaf643-istiod-ca-cert\") pod \"e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89\" (UID: \"bcf12e46-656a-4d89-962a-b1a6cefaf643\") " pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" Sep 11 12:05:06.086200 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:06.086027 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcf12e46-656a-4d89-962a-b1a6cefaf643-istio-token\") pod \"e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89\" (UID: \"bcf12e46-656a-4d89-962a-b1a6cefaf643\") " pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" Sep 11 12:05:06.086200 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:06.086056 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcf12e46-656a-4d89-962a-b1a6cefaf643-istio-envoy\") pod \"e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89\" (UID: \"bcf12e46-656a-4d89-962a-b1a6cefaf643\") " pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" Sep 11 12:05:06.086200 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:06.086093 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf12e46-656a-4d89-962a-b1a6cefaf643-workload-socket\") pod \"e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89\" (UID: \"bcf12e46-656a-4d89-962a-b1a6cefaf643\") " pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" Sep 11 12:05:06.086200 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:06.086122 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcf12e46-656a-4d89-962a-b1a6cefaf643-istio-data\") pod \"e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89\" (UID: \"bcf12e46-656a-4d89-962a-b1a6cefaf643\") " pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" Sep 11 12:05:06.086200 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:06.086150 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf12e46-656a-4d89-962a-b1a6cefaf643-credential-socket\") pod \"e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89\" (UID: \"bcf12e46-656a-4d89-962a-b1a6cefaf643\") " pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" Sep 11 12:05:06.086200 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:06.086173 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcf12e46-656a-4d89-962a-b1a6cefaf643-workload-certs\") pod \"e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89\" (UID: \"bcf12e46-656a-4d89-962a-b1a6cefaf643\") " pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" Sep 11 12:05:06.086539 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:06.086229 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcf12e46-656a-4d89-962a-b1a6cefaf643-istio-podinfo\") pod \"e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89\" (UID: \"bcf12e46-656a-4d89-962a-b1a6cefaf643\") " pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" Sep 11 12:05:06.086622 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:06.086559 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcf12e46-656a-4d89-962a-b1a6cefaf643-istio-data\") pod \"e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89\" (UID: \"bcf12e46-656a-4d89-962a-b1a6cefaf643\") " pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" Sep 11 12:05:06.086678 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:06.086611 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf12e46-656a-4d89-962a-b1a6cefaf643-workload-socket\") pod \"e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89\" (UID: \"bcf12e46-656a-4d89-962a-b1a6cefaf643\") " pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" Sep 11 12:05:06.086678 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:06.086664 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcf12e46-656a-4d89-962a-b1a6cefaf643-workload-certs\") pod \"e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89\" (UID: \"bcf12e46-656a-4d89-962a-b1a6cefaf643\") " pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" Sep 11 12:05:06.086887 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:06.086698 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf12e46-656a-4d89-962a-b1a6cefaf643-credential-socket\") pod \"e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89\" (UID: \"bcf12e46-656a-4d89-962a-b1a6cefaf643\") " pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" Sep 11 12:05:06.086887 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:06.086773 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcf12e46-656a-4d89-962a-b1a6cefaf643-istiod-ca-cert\") pod \"e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89\" (UID: \"bcf12e46-656a-4d89-962a-b1a6cefaf643\") " pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" Sep 11 12:05:06.086887 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:06.086787 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6fs8\" (UniqueName: \"kubernetes.io/projected/bcf12e46-656a-4d89-962a-b1a6cefaf643-kube-api-access-x6fs8\") pod \"e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89\" (UID: \"bcf12e46-656a-4d89-962a-b1a6cefaf643\") " pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" Sep 11 12:05:06.088528 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:06.088472 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcf12e46-656a-4d89-962a-b1a6cefaf643-istio-envoy\") pod \"e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89\" (UID: \"bcf12e46-656a-4d89-962a-b1a6cefaf643\") " pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" Sep 11 12:05:06.088774 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:06.088754 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcf12e46-656a-4d89-962a-b1a6cefaf643-istio-podinfo\") pod \"e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89\" (UID: \"bcf12e46-656a-4d89-962a-b1a6cefaf643\") " pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" Sep 11 12:05:06.103283 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:06.103254 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcf12e46-656a-4d89-962a-b1a6cefaf643-istio-token\") pod \"e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89\" (UID: \"bcf12e46-656a-4d89-962a-b1a6cefaf643\") " pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" Sep 11 12:05:06.104117 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:06.104095 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6fs8\" (UniqueName: \"kubernetes.io/projected/bcf12e46-656a-4d89-962a-b1a6cefaf643-kube-api-access-x6fs8\") pod \"e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89\" (UID: \"bcf12e46-656a-4d89-962a-b1a6cefaf643\") " pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" Sep 11 12:05:06.150007 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:06.149966 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" Sep 11 12:05:06.437019 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:06.436991 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89"] Sep 11 12:05:06.438605 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:05:06.438581 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcf12e46_656a_4d89_962a_b1a6cefaf643.slice/crio-f3f1ebad78671732f1e0dca54bf58f6e7de8a9f7d623f472937257455614401d WatchSource:0}: Error finding container f3f1ebad78671732f1e0dca54bf58f6e7de8a9f7d623f472937257455614401d: Status 404 returned error can't find the container with id f3f1ebad78671732f1e0dca54bf58f6e7de8a9f7d623f472937257455614401d Sep 11 12:05:06.440925 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:06.440891 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:05:06.440994 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:06.440967 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:05:06.441032 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:06.440994 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:05:07.062953 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:07.062912 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" event={"ID":"bcf12e46-656a-4d89-962a-b1a6cefaf643","Type":"ContainerStarted","Data":"a659c81d7f9855213e2684fe4963da0856a21dd5d2c0a2b6c3e8773285b2220e"} Sep 11 12:05:07.063147 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:07.062979 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" event={"ID":"bcf12e46-656a-4d89-962a-b1a6cefaf643","Type":"ContainerStarted","Data":"f3f1ebad78671732f1e0dca54bf58f6e7de8a9f7d623f472937257455614401d"} Sep 11 12:05:07.099587 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:07.099529 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" podStartSLOduration=2.099512334 podStartE2EDuration="2.099512334s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:05:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:05:07.097888325 +0000 UTC m=+1038.862861491" watchObservedRunningTime="2026-09-11 12:05:07.099512334 +0000 UTC m=+1038.864485512" Sep 11 12:05:07.150473 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:07.150439 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" Sep 11 12:05:07.155508 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:07.155485 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" Sep 11 12:05:08.067567 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:08.067533 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" Sep 11 12:05:08.068539 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:08.068519 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" Sep 11 12:05:10.397530 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:10.397505 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" Sep 11 12:05:10.528070 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:10.527992 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j8p4c\" (UniqueName: \"kubernetes.io/projected/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-kube-api-access-j8p4c\") pod \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\" (UID: \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\") " Sep 11 12:05:10.528070 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:10.528041 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-workload-socket\") pod \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\" (UID: \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\") " Sep 11 12:05:10.528264 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:10.528085 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-workload-certs\") pod \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\" (UID: \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\") " Sep 11 12:05:10.528264 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:10.528109 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-istio-data\") pod \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\" (UID: \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\") " Sep 11 12:05:10.528264 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:10.528151 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-credential-socket\") pod \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\" (UID: \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\") " Sep 11 12:05:10.528264 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:10.528190 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-istio-podinfo\") pod \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\" (UID: \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\") " Sep 11 12:05:10.528264 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:10.528214 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-istio-token\") pod \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\" (UID: \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\") " Sep 11 12:05:10.528264 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:10.528258 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-istiod-ca-cert\") pod \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\" (UID: \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\") " Sep 11 12:05:10.528569 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:10.528359 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-istio-envoy\") pod \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\" (UID: \"3d23b88e-3a6a-472e-ab1e-904ca680e5f4\") " Sep 11 12:05:10.528569 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:10.528386 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3d23b88e-3a6a-472e-ab1e-904ca680e5f4" (UID: "3d23b88e-3a6a-472e-ab1e-904ca680e5f4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:05:10.528569 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:10.528396 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3d23b88e-3a6a-472e-ab1e-904ca680e5f4" (UID: "3d23b88e-3a6a-472e-ab1e-904ca680e5f4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:05:10.528569 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:10.528455 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3d23b88e-3a6a-472e-ab1e-904ca680e5f4" (UID: "3d23b88e-3a6a-472e-ab1e-904ca680e5f4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:05:10.528569 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:10.528462 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-istio-data" (OuterVolumeSpecName: "istio-data") pod "3d23b88e-3a6a-472e-ab1e-904ca680e5f4" (UID: "3d23b88e-3a6a-472e-ab1e-904ca680e5f4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:05:10.528833 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:10.528750 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-workload-socket\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:05:10.528833 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:10.528772 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-workload-certs\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:05:10.528833 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:10.528770 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3d23b88e-3a6a-472e-ab1e-904ca680e5f4" (UID: "3d23b88e-3a6a-472e-ab1e-904ca680e5f4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 12:05:10.528833 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:10.528785 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-istio-data\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:05:10.528833 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:10.528799 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-credential-socket\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:05:10.530381 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:10.530360 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-kube-api-access-j8p4c" (OuterVolumeSpecName: "kube-api-access-j8p4c") pod "3d23b88e-3a6a-472e-ab1e-904ca680e5f4" (UID: "3d23b88e-3a6a-472e-ab1e-904ca680e5f4"). InnerVolumeSpecName "kube-api-access-j8p4c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:05:10.530880 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:10.530847 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3d23b88e-3a6a-472e-ab1e-904ca680e5f4" (UID: "3d23b88e-3a6a-472e-ab1e-904ca680e5f4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 11 12:05:10.530880 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:10.530853 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-istio-token" (OuterVolumeSpecName: "istio-token") pod "3d23b88e-3a6a-472e-ab1e-904ca680e5f4" (UID: "3d23b88e-3a6a-472e-ab1e-904ca680e5f4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:05:10.531017 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:10.530973 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3d23b88e-3a6a-472e-ab1e-904ca680e5f4" (UID: "3d23b88e-3a6a-472e-ab1e-904ca680e5f4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:05:10.629927 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:10.629837 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-istio-envoy\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:05:10.630309 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:10.629939 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j8p4c\" (UniqueName: \"kubernetes.io/projected/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-kube-api-access-j8p4c\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:05:10.630309 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:10.629962 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-istio-podinfo\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:05:10.630309 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:10.629976 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-istio-token\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:05:10.630309 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:10.629997 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d23b88e-3a6a-472e-ab1e-904ca680e5f4-istiod-ca-cert\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:05:11.084945 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:11.084906 2566 generic.go:356] "Generic (PLEG): container finished" podID="3d23b88e-3a6a-472e-ab1e-904ca680e5f4" containerID="b66376aaa1ab8ef03637fb42563e60810461884a18dfce6f5f43589d823b0120" exitCode=0 Sep 11 12:05:11.085118 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:11.084961 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" Sep 11 12:05:11.085118 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:11.084972 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" event={"ID":"3d23b88e-3a6a-472e-ab1e-904ca680e5f4","Type":"ContainerDied","Data":"b66376aaa1ab8ef03637fb42563e60810461884a18dfce6f5f43589d823b0120"} Sep 11 12:05:11.085118 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:11.085023 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc" event={"ID":"3d23b88e-3a6a-472e-ab1e-904ca680e5f4","Type":"ContainerDied","Data":"f24425d26665820580f2cf957a3a5dea93063891ae3ef2a50e82d8c4712f8324"} Sep 11 12:05:11.085118 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:11.085056 2566 scope.go:117] "RemoveContainer" containerID="b66376aaa1ab8ef03637fb42563e60810461884a18dfce6f5f43589d823b0120" Sep 11 12:05:11.095422 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:11.095342 2566 scope.go:117] "RemoveContainer" containerID="a68fa1e6673ebb6d21e784f69e7ae1231dcf05ca7a23bff4ca772b8b7efe1aa7" Sep 11 12:05:11.106135 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:11.106106 2566 scope.go:117] "RemoveContainer" containerID="b66376aaa1ab8ef03637fb42563e60810461884a18dfce6f5f43589d823b0120" Sep 11 12:05:11.106464 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:05:11.106441 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b66376aaa1ab8ef03637fb42563e60810461884a18dfce6f5f43589d823b0120\": container with ID starting with b66376aaa1ab8ef03637fb42563e60810461884a18dfce6f5f43589d823b0120 not found: ID does not exist" containerID="b66376aaa1ab8ef03637fb42563e60810461884a18dfce6f5f43589d823b0120" Sep 11 12:05:11.106545 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:11.106471 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b66376aaa1ab8ef03637fb42563e60810461884a18dfce6f5f43589d823b0120"} err="failed to get container status \"b66376aaa1ab8ef03637fb42563e60810461884a18dfce6f5f43589d823b0120\": rpc error: code = NotFound desc = could not find container \"b66376aaa1ab8ef03637fb42563e60810461884a18dfce6f5f43589d823b0120\": container with ID starting with b66376aaa1ab8ef03637fb42563e60810461884a18dfce6f5f43589d823b0120 not found: ID does not exist" Sep 11 12:05:11.106545 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:11.106490 2566 scope.go:117] "RemoveContainer" containerID="a68fa1e6673ebb6d21e784f69e7ae1231dcf05ca7a23bff4ca772b8b7efe1aa7" Sep 11 12:05:11.106793 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:05:11.106775 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a68fa1e6673ebb6d21e784f69e7ae1231dcf05ca7a23bff4ca772b8b7efe1aa7\": container with ID starting with a68fa1e6673ebb6d21e784f69e7ae1231dcf05ca7a23bff4ca772b8b7efe1aa7 not found: ID does not exist" containerID="a68fa1e6673ebb6d21e784f69e7ae1231dcf05ca7a23bff4ca772b8b7efe1aa7" Sep 11 12:05:11.106833 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:11.106802 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a68fa1e6673ebb6d21e784f69e7ae1231dcf05ca7a23bff4ca772b8b7efe1aa7"} err="failed to get container status \"a68fa1e6673ebb6d21e784f69e7ae1231dcf05ca7a23bff4ca772b8b7efe1aa7\": rpc error: code = NotFound desc = could not find container \"a68fa1e6673ebb6d21e784f69e7ae1231dcf05ca7a23bff4ca772b8b7efe1aa7\": container with ID starting with a68fa1e6673ebb6d21e784f69e7ae1231dcf05ca7a23bff4ca772b8b7efe1aa7 not found: ID does not exist" Sep 11 12:05:11.110339 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:11.110315 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc"] Sep 11 12:05:11.125954 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:11.125914 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-cdec04-openshift-default-76fcf7b4bd-42jwc"] Sep 11 12:05:11.651777 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:11.651747 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-69b1ea-74dd4478d5-qdt5j"] Sep 11 12:05:11.652165 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:11.652121 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d23b88e-3a6a-472e-ab1e-904ca680e5f4" containerName="istio-proxy" Sep 11 12:05:11.652165 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:11.652133 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d23b88e-3a6a-472e-ab1e-904ca680e5f4" containerName="istio-proxy" Sep 11 12:05:11.652165 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:11.652147 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d23b88e-3a6a-472e-ab1e-904ca680e5f4" containerName="istio-proxy" Sep 11 12:05:11.652165 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:11.652152 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d23b88e-3a6a-472e-ab1e-904ca680e5f4" containerName="istio-proxy" Sep 11 12:05:11.652165 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:11.652163 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a" containerName="istio-proxy" Sep 11 12:05:11.652165 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:11.652168 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0b95a4e-4b47-48da-bb25-a9e7b9b0de1a" containerName="istio-proxy" Sep 11 12:05:11.652424 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:11.652214 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d23b88e-3a6a-472e-ab1e-904ca680e5f4" containerName="istio-proxy" Sep 11 12:05:11.655234 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:11.655215 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-69b1ea-74dd4478d5-qdt5j" Sep 11 12:05:11.658577 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:11.658556 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-69b1ea\"" Sep 11 12:05:11.659032 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:11.659008 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-69b1ea-dockercfg-qjpsr\"" Sep 11 12:05:11.673368 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:11.673347 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-69b1ea-74dd4478d5-qdt5j"] Sep 11 12:05:11.739646 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:11.739613 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-js2pc\" (UniqueName: \"kubernetes.io/projected/332c2183-9cf2-4685-a2c2-401af97223a6-kube-api-access-js2pc\") pod \"payload-processing-e2e-explicit-69b1ea-74dd4478d5-qdt5j\" (UID: \"332c2183-9cf2-4685-a2c2-401af97223a6\") " pod="openshift-ingress/payload-processing-e2e-explicit-69b1ea-74dd4478d5-qdt5j" Sep 11 12:05:11.739823 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:11.739702 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/332c2183-9cf2-4685-a2c2-401af97223a6-plugins-config-volume\") pod \"payload-processing-e2e-explicit-69b1ea-74dd4478d5-qdt5j\" (UID: \"332c2183-9cf2-4685-a2c2-401af97223a6\") " pod="openshift-ingress/payload-processing-e2e-explicit-69b1ea-74dd4478d5-qdt5j" Sep 11 12:05:11.840722 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:11.840687 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js2pc\" (UniqueName: \"kubernetes.io/projected/332c2183-9cf2-4685-a2c2-401af97223a6-kube-api-access-js2pc\") pod \"payload-processing-e2e-explicit-69b1ea-74dd4478d5-qdt5j\" (UID: \"332c2183-9cf2-4685-a2c2-401af97223a6\") " pod="openshift-ingress/payload-processing-e2e-explicit-69b1ea-74dd4478d5-qdt5j" Sep 11 12:05:11.841371 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:11.840775 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/332c2183-9cf2-4685-a2c2-401af97223a6-plugins-config-volume\") pod \"payload-processing-e2e-explicit-69b1ea-74dd4478d5-qdt5j\" (UID: \"332c2183-9cf2-4685-a2c2-401af97223a6\") " pod="openshift-ingress/payload-processing-e2e-explicit-69b1ea-74dd4478d5-qdt5j" Sep 11 12:05:11.841682 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:11.841656 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/332c2183-9cf2-4685-a2c2-401af97223a6-plugins-config-volume\") pod \"payload-processing-e2e-explicit-69b1ea-74dd4478d5-qdt5j\" (UID: \"332c2183-9cf2-4685-a2c2-401af97223a6\") " pod="openshift-ingress/payload-processing-e2e-explicit-69b1ea-74dd4478d5-qdt5j" Sep 11 12:05:11.861189 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:11.861166 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-js2pc\" (UniqueName: \"kubernetes.io/projected/332c2183-9cf2-4685-a2c2-401af97223a6-kube-api-access-js2pc\") pod \"payload-processing-e2e-explicit-69b1ea-74dd4478d5-qdt5j\" (UID: \"332c2183-9cf2-4685-a2c2-401af97223a6\") " pod="openshift-ingress/payload-processing-e2e-explicit-69b1ea-74dd4478d5-qdt5j" Sep 11 12:05:11.966028 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:11.965925 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-69b1ea-74dd4478d5-qdt5j" Sep 11 12:05:12.247999 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:05:12.247963 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod332c2183_9cf2_4685_a2c2_401af97223a6.slice/crio-f356a5de91d968ce4c13bab8702f97d0fa127f12ff1059d3288879892d6f8a02 WatchSource:0}: Error finding container f356a5de91d968ce4c13bab8702f97d0fa127f12ff1059d3288879892d6f8a02: Status 404 returned error can't find the container with id f356a5de91d968ce4c13bab8702f97d0fa127f12ff1059d3288879892d6f8a02 Sep 11 12:05:12.250179 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:12.250157 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-69b1ea-74dd4478d5-qdt5j"] Sep 11 12:05:12.758776 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:12.758741 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d23b88e-3a6a-472e-ab1e-904ca680e5f4" path="/var/lib/kubelet/pods/3d23b88e-3a6a-472e-ab1e-904ca680e5f4/volumes" Sep 11 12:05:13.098282 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:13.098243 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-69b1ea-74dd4478d5-qdt5j" event={"ID":"332c2183-9cf2-4685-a2c2-401af97223a6","Type":"ContainerStarted","Data":"5a934c188ebdcc5314fd67b153a8f3fdfa595b7c616e0e1fa59d61b96e4d1dad"} Sep 11 12:05:13.098445 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:13.098311 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-69b1ea-74dd4478d5-qdt5j" event={"ID":"332c2183-9cf2-4685-a2c2-401af97223a6","Type":"ContainerStarted","Data":"f356a5de91d968ce4c13bab8702f97d0fa127f12ff1059d3288879892d6f8a02"} Sep 11 12:05:13.098445 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:13.098354 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-69b1ea-74dd4478d5-qdt5j" Sep 11 12:05:24.105789 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:24.105754 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-69b1ea-74dd4478d5-qdt5j" Sep 11 12:05:24.139819 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:24.139763 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-69b1ea-74dd4478d5-qdt5j" podStartSLOduration=13.139746118 podStartE2EDuration="13.139746118s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:05:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:05:13.125373037 +0000 UTC m=+1044.890346230" watchObservedRunningTime="2026-09-11 12:05:24.139746118 +0000 UTC m=+1055.904719292" Sep 11 12:05:27.168767 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:27.168732 2566 generic.go:356] "Generic (PLEG): container finished" podID="bcf12e46-656a-4d89-962a-b1a6cefaf643" containerID="a659c81d7f9855213e2684fe4963da0856a21dd5d2c0a2b6c3e8773285b2220e" exitCode=0 Sep 11 12:05:27.169178 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:27.168815 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" event={"ID":"bcf12e46-656a-4d89-962a-b1a6cefaf643","Type":"ContainerDied","Data":"a659c81d7f9855213e2684fe4963da0856a21dd5d2c0a2b6c3e8773285b2220e"} Sep 11 12:05:27.169244 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:27.169230 2566 scope.go:117] "RemoveContainer" containerID="a659c81d7f9855213e2684fe4963da0856a21dd5d2c0a2b6c3e8773285b2220e" Sep 11 12:05:27.170332 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:27.170301 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:05:27.170396 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:27.170377 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:05:27.170430 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:27.170419 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:05:28.176037 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:28.176000 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" event={"ID":"bcf12e46-656a-4d89-962a-b1a6cefaf643","Type":"ContainerStarted","Data":"517a67755a25f2ca758b594e48a19104a54e62d6248f91818179aaa0c279be58"} Sep 11 12:05:29.150830 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:29.150790 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" Sep 11 12:05:29.151110 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:29.151084 2566 patch_prober.go:28] interesting pod/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Sep 11 12:05:29.151205 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:29.151142 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" podUID="bcf12e46-656a-4d89-962a-b1a6cefaf643" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Sep 11 12:05:30.150600 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:30.150557 2566 patch_prober.go:28] interesting pod/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Sep 11 12:05:30.151023 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:30.150613 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" podUID="bcf12e46-656a-4d89-962a-b1a6cefaf643" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Sep 11 12:05:31.150757 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:31.150717 2566 patch_prober.go:28] interesting pod/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Sep 11 12:05:31.151266 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:31.150799 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" podUID="bcf12e46-656a-4d89-962a-b1a6cefaf643" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Sep 11 12:05:32.151284 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:32.151244 2566 patch_prober.go:28] interesting pod/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Sep 11 12:05:32.151676 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:32.151304 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" podUID="bcf12e46-656a-4d89-962a-b1a6cefaf643" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Sep 11 12:05:33.151221 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:33.151178 2566 patch_prober.go:28] interesting pod/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Sep 11 12:05:33.151441 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:33.151256 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" podUID="bcf12e46-656a-4d89-962a-b1a6cefaf643" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Sep 11 12:05:34.155302 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:34.155270 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" Sep 11 12:05:34.155746 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:34.155567 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" Sep 11 12:05:34.156261 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:05:34.156241 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" Sep 11 12:06:01.717303 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:01.717269 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-69b1ea-74dd4478d5-qdt5j"] Sep 11 12:06:01.717809 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:01.717552 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-69b1ea-74dd4478d5-qdt5j" podUID="332c2183-9cf2-4685-a2c2-401af97223a6" containerName="payload-processing" containerID="cri-o://5a934c188ebdcc5314fd67b153a8f3fdfa595b7c616e0e1fa59d61b96e4d1dad" gracePeriod=30 Sep 11 12:06:01.975946 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:01.975882 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-69b1ea-74dd4478d5-qdt5j" Sep 11 12:06:02.018040 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:02.018002 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-js2pc\" (UniqueName: \"kubernetes.io/projected/332c2183-9cf2-4685-a2c2-401af97223a6-kube-api-access-js2pc\") pod \"332c2183-9cf2-4685-a2c2-401af97223a6\" (UID: \"332c2183-9cf2-4685-a2c2-401af97223a6\") " Sep 11 12:06:02.018194 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:02.018121 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/332c2183-9cf2-4685-a2c2-401af97223a6-plugins-config-volume\") pod \"332c2183-9cf2-4685-a2c2-401af97223a6\" (UID: \"332c2183-9cf2-4685-a2c2-401af97223a6\") " Sep 11 12:06:02.018481 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:02.018456 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/332c2183-9cf2-4685-a2c2-401af97223a6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "332c2183-9cf2-4685-a2c2-401af97223a6" (UID: "332c2183-9cf2-4685-a2c2-401af97223a6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 12:06:02.020132 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:02.020107 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/332c2183-9cf2-4685-a2c2-401af97223a6-kube-api-access-js2pc" (OuterVolumeSpecName: "kube-api-access-js2pc") pod "332c2183-9cf2-4685-a2c2-401af97223a6" (UID: "332c2183-9cf2-4685-a2c2-401af97223a6"). InnerVolumeSpecName "kube-api-access-js2pc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:06:02.119282 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:02.119240 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-js2pc\" (UniqueName: \"kubernetes.io/projected/332c2183-9cf2-4685-a2c2-401af97223a6-kube-api-access-js2pc\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:06:02.119282 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:02.119274 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/332c2183-9cf2-4685-a2c2-401af97223a6-plugins-config-volume\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:06:02.343053 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:02.343005 2566 generic.go:356] "Generic (PLEG): container finished" podID="332c2183-9cf2-4685-a2c2-401af97223a6" containerID="5a934c188ebdcc5314fd67b153a8f3fdfa595b7c616e0e1fa59d61b96e4d1dad" exitCode=0 Sep 11 12:06:02.343053 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:02.343053 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-69b1ea-74dd4478d5-qdt5j" Sep 11 12:06:02.343277 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:02.343084 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-69b1ea-74dd4478d5-qdt5j" event={"ID":"332c2183-9cf2-4685-a2c2-401af97223a6","Type":"ContainerDied","Data":"5a934c188ebdcc5314fd67b153a8f3fdfa595b7c616e0e1fa59d61b96e4d1dad"} Sep 11 12:06:02.343277 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:02.343135 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-69b1ea-74dd4478d5-qdt5j" event={"ID":"332c2183-9cf2-4685-a2c2-401af97223a6","Type":"ContainerDied","Data":"f356a5de91d968ce4c13bab8702f97d0fa127f12ff1059d3288879892d6f8a02"} Sep 11 12:06:02.343277 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:02.343157 2566 scope.go:117] "RemoveContainer" containerID="5a934c188ebdcc5314fd67b153a8f3fdfa595b7c616e0e1fa59d61b96e4d1dad" Sep 11 12:06:02.352348 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:02.352332 2566 scope.go:117] "RemoveContainer" containerID="5a934c188ebdcc5314fd67b153a8f3fdfa595b7c616e0e1fa59d61b96e4d1dad" Sep 11 12:06:02.352595 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:06:02.352576 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a934c188ebdcc5314fd67b153a8f3fdfa595b7c616e0e1fa59d61b96e4d1dad\": container with ID starting with 5a934c188ebdcc5314fd67b153a8f3fdfa595b7c616e0e1fa59d61b96e4d1dad not found: ID does not exist" containerID="5a934c188ebdcc5314fd67b153a8f3fdfa595b7c616e0e1fa59d61b96e4d1dad" Sep 11 12:06:02.352649 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:02.352605 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a934c188ebdcc5314fd67b153a8f3fdfa595b7c616e0e1fa59d61b96e4d1dad"} err="failed to get container status \"5a934c188ebdcc5314fd67b153a8f3fdfa595b7c616e0e1fa59d61b96e4d1dad\": rpc error: code = NotFound desc = could not find container \"5a934c188ebdcc5314fd67b153a8f3fdfa595b7c616e0e1fa59d61b96e4d1dad\": container with ID starting with 5a934c188ebdcc5314fd67b153a8f3fdfa595b7c616e0e1fa59d61b96e4d1dad not found: ID does not exist" Sep 11 12:06:02.365456 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:02.365430 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-69b1ea-74dd4478d5-qdt5j"] Sep 11 12:06:02.370015 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:02.369994 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-69b1ea-74dd4478d5-qdt5j"] Sep 11 12:06:02.758645 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:02.758564 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="332c2183-9cf2-4685-a2c2-401af97223a6" path="/var/lib/kubelet/pods/332c2183-9cf2-4685-a2c2-401af97223a6/volumes" Sep 11 12:06:09.637746 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:09.637711 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89"] Sep 11 12:06:09.638144 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:09.637962 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" podUID="bcf12e46-656a-4d89-962a-b1a6cefaf643" containerName="istio-proxy" containerID="cri-o://517a67755a25f2ca758b594e48a19104a54e62d6248f91818179aaa0c279be58" gracePeriod=30 Sep 11 12:06:14.891552 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:14.891525 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" Sep 11 12:06:14.949774 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:14.949679 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6fs8\" (UniqueName: \"kubernetes.io/projected/bcf12e46-656a-4d89-962a-b1a6cefaf643-kube-api-access-x6fs8\") pod \"bcf12e46-656a-4d89-962a-b1a6cefaf643\" (UID: \"bcf12e46-656a-4d89-962a-b1a6cefaf643\") " Sep 11 12:06:14.949774 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:14.949741 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcf12e46-656a-4d89-962a-b1a6cefaf643-istio-podinfo\") pod \"bcf12e46-656a-4d89-962a-b1a6cefaf643\" (UID: \"bcf12e46-656a-4d89-962a-b1a6cefaf643\") " Sep 11 12:06:14.949774 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:14.949763 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcf12e46-656a-4d89-962a-b1a6cefaf643-workload-certs\") pod \"bcf12e46-656a-4d89-962a-b1a6cefaf643\" (UID: \"bcf12e46-656a-4d89-962a-b1a6cefaf643\") " Sep 11 12:06:14.950076 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:14.949799 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcf12e46-656a-4d89-962a-b1a6cefaf643-istio-envoy\") pod \"bcf12e46-656a-4d89-962a-b1a6cefaf643\" (UID: \"bcf12e46-656a-4d89-962a-b1a6cefaf643\") " Sep 11 12:06:14.950076 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:14.949817 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcf12e46-656a-4d89-962a-b1a6cefaf643-istio-data\") pod \"bcf12e46-656a-4d89-962a-b1a6cefaf643\" (UID: \"bcf12e46-656a-4d89-962a-b1a6cefaf643\") " Sep 11 12:06:14.950076 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:14.949859 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcf12e46-656a-4d89-962a-b1a6cefaf643-istiod-ca-cert\") pod \"bcf12e46-656a-4d89-962a-b1a6cefaf643\" (UID: \"bcf12e46-656a-4d89-962a-b1a6cefaf643\") " Sep 11 12:06:14.950076 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:14.949890 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcf12e46-656a-4d89-962a-b1a6cefaf643-istio-token\") pod \"bcf12e46-656a-4d89-962a-b1a6cefaf643\" (UID: \"bcf12e46-656a-4d89-962a-b1a6cefaf643\") " Sep 11 12:06:14.950076 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:14.949952 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf12e46-656a-4d89-962a-b1a6cefaf643-credential-socket\") pod \"bcf12e46-656a-4d89-962a-b1a6cefaf643\" (UID: \"bcf12e46-656a-4d89-962a-b1a6cefaf643\") " Sep 11 12:06:14.950076 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:14.949980 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf12e46-656a-4d89-962a-b1a6cefaf643-workload-socket\") pod \"bcf12e46-656a-4d89-962a-b1a6cefaf643\" (UID: \"bcf12e46-656a-4d89-962a-b1a6cefaf643\") " Sep 11 12:06:14.950379 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:14.950125 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcf12e46-656a-4d89-962a-b1a6cefaf643-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bcf12e46-656a-4d89-962a-b1a6cefaf643" (UID: "bcf12e46-656a-4d89-962a-b1a6cefaf643"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:06:14.950379 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:14.950318 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcf12e46-656a-4d89-962a-b1a6cefaf643-workload-certs\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:06:14.950487 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:14.950384 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcf12e46-656a-4d89-962a-b1a6cefaf643-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bcf12e46-656a-4d89-962a-b1a6cefaf643" (UID: "bcf12e46-656a-4d89-962a-b1a6cefaf643"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:06:14.950487 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:14.950403 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcf12e46-656a-4d89-962a-b1a6cefaf643-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bcf12e46-656a-4d89-962a-b1a6cefaf643" (UID: "bcf12e46-656a-4d89-962a-b1a6cefaf643"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:06:14.950692 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:14.950654 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcf12e46-656a-4d89-962a-b1a6cefaf643-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bcf12e46-656a-4d89-962a-b1a6cefaf643" (UID: "bcf12e46-656a-4d89-962a-b1a6cefaf643"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 12:06:14.950692 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:14.950681 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcf12e46-656a-4d89-962a-b1a6cefaf643-istio-data" (OuterVolumeSpecName: "istio-data") pod "bcf12e46-656a-4d89-962a-b1a6cefaf643" (UID: "bcf12e46-656a-4d89-962a-b1a6cefaf643"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:06:14.952134 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:14.952106 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bcf12e46-656a-4d89-962a-b1a6cefaf643-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bcf12e46-656a-4d89-962a-b1a6cefaf643" (UID: "bcf12e46-656a-4d89-962a-b1a6cefaf643"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 11 12:06:14.952134 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:14.952118 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcf12e46-656a-4d89-962a-b1a6cefaf643-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bcf12e46-656a-4d89-962a-b1a6cefaf643" (UID: "bcf12e46-656a-4d89-962a-b1a6cefaf643"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:06:14.952291 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:14.952113 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcf12e46-656a-4d89-962a-b1a6cefaf643-istio-token" (OuterVolumeSpecName: "istio-token") pod "bcf12e46-656a-4d89-962a-b1a6cefaf643" (UID: "bcf12e46-656a-4d89-962a-b1a6cefaf643"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:06:14.952417 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:14.952396 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcf12e46-656a-4d89-962a-b1a6cefaf643-kube-api-access-x6fs8" (OuterVolumeSpecName: "kube-api-access-x6fs8") pod "bcf12e46-656a-4d89-962a-b1a6cefaf643" (UID: "bcf12e46-656a-4d89-962a-b1a6cefaf643"). InnerVolumeSpecName "kube-api-access-x6fs8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:06:15.051796 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:15.051763 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcf12e46-656a-4d89-962a-b1a6cefaf643-istio-podinfo\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:06:15.051796 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:15.051791 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcf12e46-656a-4d89-962a-b1a6cefaf643-istio-envoy\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:06:15.051796 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:15.051803 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcf12e46-656a-4d89-962a-b1a6cefaf643-istio-data\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:06:15.052065 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:15.051813 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcf12e46-656a-4d89-962a-b1a6cefaf643-istiod-ca-cert\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:06:15.052065 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:15.051820 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcf12e46-656a-4d89-962a-b1a6cefaf643-istio-token\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:06:15.052065 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:15.051829 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf12e46-656a-4d89-962a-b1a6cefaf643-credential-socket\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:06:15.052065 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:15.051837 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf12e46-656a-4d89-962a-b1a6cefaf643-workload-socket\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:06:15.052065 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:15.051846 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6fs8\" (UniqueName: \"kubernetes.io/projected/bcf12e46-656a-4d89-962a-b1a6cefaf643-kube-api-access-x6fs8\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:06:15.409926 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:15.409880 2566 generic.go:356] "Generic (PLEG): container finished" podID="bcf12e46-656a-4d89-962a-b1a6cefaf643" containerID="517a67755a25f2ca758b594e48a19104a54e62d6248f91818179aaa0c279be58" exitCode=0 Sep 11 12:06:15.409926 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:15.409903 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" event={"ID":"bcf12e46-656a-4d89-962a-b1a6cefaf643","Type":"ContainerDied","Data":"517a67755a25f2ca758b594e48a19104a54e62d6248f91818179aaa0c279be58"} Sep 11 12:06:15.409926 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:15.409924 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" Sep 11 12:06:15.410227 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:15.409966 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89" event={"ID":"bcf12e46-656a-4d89-962a-b1a6cefaf643","Type":"ContainerDied","Data":"f3f1ebad78671732f1e0dca54bf58f6e7de8a9f7d623f472937257455614401d"} Sep 11 12:06:15.410227 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:15.409989 2566 scope.go:117] "RemoveContainer" containerID="517a67755a25f2ca758b594e48a19104a54e62d6248f91818179aaa0c279be58" Sep 11 12:06:15.419627 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:15.419607 2566 scope.go:117] "RemoveContainer" containerID="a659c81d7f9855213e2684fe4963da0856a21dd5d2c0a2b6c3e8773285b2220e" Sep 11 12:06:15.427372 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:15.427348 2566 scope.go:117] "RemoveContainer" containerID="517a67755a25f2ca758b594e48a19104a54e62d6248f91818179aaa0c279be58" Sep 11 12:06:15.427653 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:06:15.427634 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"517a67755a25f2ca758b594e48a19104a54e62d6248f91818179aaa0c279be58\": container with ID starting with 517a67755a25f2ca758b594e48a19104a54e62d6248f91818179aaa0c279be58 not found: ID does not exist" containerID="517a67755a25f2ca758b594e48a19104a54e62d6248f91818179aaa0c279be58" Sep 11 12:06:15.427704 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:15.427661 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"517a67755a25f2ca758b594e48a19104a54e62d6248f91818179aaa0c279be58"} err="failed to get container status \"517a67755a25f2ca758b594e48a19104a54e62d6248f91818179aaa0c279be58\": rpc error: code = NotFound desc = could not find container \"517a67755a25f2ca758b594e48a19104a54e62d6248f91818179aaa0c279be58\": container with ID starting with 517a67755a25f2ca758b594e48a19104a54e62d6248f91818179aaa0c279be58 not found: ID does not exist" Sep 11 12:06:15.427704 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:15.427678 2566 scope.go:117] "RemoveContainer" containerID="a659c81d7f9855213e2684fe4963da0856a21dd5d2c0a2b6c3e8773285b2220e" Sep 11 12:06:15.427946 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:06:15.427930 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a659c81d7f9855213e2684fe4963da0856a21dd5d2c0a2b6c3e8773285b2220e\": container with ID starting with a659c81d7f9855213e2684fe4963da0856a21dd5d2c0a2b6c3e8773285b2220e not found: ID does not exist" containerID="a659c81d7f9855213e2684fe4963da0856a21dd5d2c0a2b6c3e8773285b2220e" Sep 11 12:06:15.427995 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:15.427949 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a659c81d7f9855213e2684fe4963da0856a21dd5d2c0a2b6c3e8773285b2220e"} err="failed to get container status \"a659c81d7f9855213e2684fe4963da0856a21dd5d2c0a2b6c3e8773285b2220e\": rpc error: code = NotFound desc = could not find container \"a659c81d7f9855213e2684fe4963da0856a21dd5d2c0a2b6c3e8773285b2220e\": container with ID starting with a659c81d7f9855213e2684fe4963da0856a21dd5d2c0a2b6c3e8773285b2220e not found: ID does not exist" Sep 11 12:06:15.448153 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:15.448119 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89"] Sep 11 12:06:15.456396 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:15.456367 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-69b1ea-openshift-default-57858dbb96-ntl89"] Sep 11 12:06:16.759221 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:16.759188 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcf12e46-656a-4d89-962a-b1a6cefaf643" path="/var/lib/kubelet/pods/bcf12e46-656a-4d89-962a-b1a6cefaf643/volumes" Sep 11 12:06:50.054306 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.054265 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9"] Sep 11 12:06:50.055599 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.055537 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9" podUID="58351aff-8939-4449-9523-e07001ad0840" containerName="maas-api" containerID="cri-o://cc54ae46fd72abf3a4fa7bd4009de713a9c29d58a4dabe56063feefce6b03655" gracePeriod=30 Sep 11 12:06:50.133705 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.133670 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-cbc380-7b4d557d8b-52rrl"] Sep 11 12:06:50.133965 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.133915 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cbc380-7b4d557d8b-52rrl" podUID="431e293a-0b82-4210-aa0d-e7ed0bab4d00" containerName="payload-pre-processing" containerID="cri-o://21181a6c7d819c51bc5f3f76894f876fbf5eeb67818119b4ec73fa2d45e46271" gracePeriod=30 Sep 11 12:06:50.400478 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.400442 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9" Sep 11 12:06:50.452359 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.452309 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cbc380-7b4d557d8b-52rrl" Sep 11 12:06:50.458916 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.458854 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58351aff-8939-4449-9523-e07001ad0840-maas-api-tls\") pod \"58351aff-8939-4449-9523-e07001ad0840\" (UID: \"58351aff-8939-4449-9523-e07001ad0840\") " Sep 11 12:06:50.459061 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.458980 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cjmgd\" (UniqueName: \"kubernetes.io/projected/58351aff-8939-4449-9523-e07001ad0840-kube-api-access-cjmgd\") pod \"58351aff-8939-4449-9523-e07001ad0840\" (UID: \"58351aff-8939-4449-9523-e07001ad0840\") " Sep 11 12:06:50.459061 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.459050 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58351aff-8939-4449-9523-e07001ad0840-metrics-certs\") pod \"58351aff-8939-4449-9523-e07001ad0840\" (UID: \"58351aff-8939-4449-9523-e07001ad0840\") " Sep 11 12:06:50.459191 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.459102 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58351aff-8939-4449-9523-e07001ad0840-openshift-service-ca\") pod \"58351aff-8939-4449-9523-e07001ad0840\" (UID: \"58351aff-8939-4449-9523-e07001ad0840\") " Sep 11 12:06:50.459488 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.459446 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58351aff-8939-4449-9523-e07001ad0840-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "58351aff-8939-4449-9523-e07001ad0840" (UID: "58351aff-8939-4449-9523-e07001ad0840"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 12:06:50.461274 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.461250 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58351aff-8939-4449-9523-e07001ad0840-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "58351aff-8939-4449-9523-e07001ad0840" (UID: "58351aff-8939-4449-9523-e07001ad0840"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 12:06:50.461371 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.461266 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58351aff-8939-4449-9523-e07001ad0840-kube-api-access-cjmgd" (OuterVolumeSpecName: "kube-api-access-cjmgd") pod "58351aff-8939-4449-9523-e07001ad0840" (UID: "58351aff-8939-4449-9523-e07001ad0840"). InnerVolumeSpecName "kube-api-access-cjmgd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:06:50.461371 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.461323 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58351aff-8939-4449-9523-e07001ad0840-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "58351aff-8939-4449-9523-e07001ad0840" (UID: "58351aff-8939-4449-9523-e07001ad0840"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 12:06:50.560726 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.560688 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8fcpb\" (UniqueName: \"kubernetes.io/projected/431e293a-0b82-4210-aa0d-e7ed0bab4d00-kube-api-access-8fcpb\") pod \"431e293a-0b82-4210-aa0d-e7ed0bab4d00\" (UID: \"431e293a-0b82-4210-aa0d-e7ed0bab4d00\") " Sep 11 12:06:50.560946 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.560762 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/431e293a-0b82-4210-aa0d-e7ed0bab4d00-plugins-config-volume\") pod \"431e293a-0b82-4210-aa0d-e7ed0bab4d00\" (UID: \"431e293a-0b82-4210-aa0d-e7ed0bab4d00\") " Sep 11 12:06:50.560946 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.560917 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cjmgd\" (UniqueName: \"kubernetes.io/projected/58351aff-8939-4449-9523-e07001ad0840-kube-api-access-cjmgd\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:06:50.560946 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.560929 2566 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58351aff-8939-4449-9523-e07001ad0840-metrics-certs\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:06:50.560946 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.560940 2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58351aff-8939-4449-9523-e07001ad0840-openshift-service-ca\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:06:50.560946 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.560948 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58351aff-8939-4449-9523-e07001ad0840-maas-api-tls\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:06:50.561545 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.561212 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/431e293a-0b82-4210-aa0d-e7ed0bab4d00-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "431e293a-0b82-4210-aa0d-e7ed0bab4d00" (UID: "431e293a-0b82-4210-aa0d-e7ed0bab4d00"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 12:06:50.562777 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.562754 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/431e293a-0b82-4210-aa0d-e7ed0bab4d00-kube-api-access-8fcpb" (OuterVolumeSpecName: "kube-api-access-8fcpb") pod "431e293a-0b82-4210-aa0d-e7ed0bab4d00" (UID: "431e293a-0b82-4210-aa0d-e7ed0bab4d00"). InnerVolumeSpecName "kube-api-access-8fcpb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:06:50.579293 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.579209 2566 generic.go:356] "Generic (PLEG): container finished" podID="431e293a-0b82-4210-aa0d-e7ed0bab4d00" containerID="21181a6c7d819c51bc5f3f76894f876fbf5eeb67818119b4ec73fa2d45e46271" exitCode=0 Sep 11 12:06:50.579293 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.579258 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cbc380-7b4d557d8b-52rrl" Sep 11 12:06:50.579507 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.579301 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cbc380-7b4d557d8b-52rrl" event={"ID":"431e293a-0b82-4210-aa0d-e7ed0bab4d00","Type":"ContainerDied","Data":"21181a6c7d819c51bc5f3f76894f876fbf5eeb67818119b4ec73fa2d45e46271"} Sep 11 12:06:50.579507 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.579364 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cbc380-7b4d557d8b-52rrl" event={"ID":"431e293a-0b82-4210-aa0d-e7ed0bab4d00","Type":"ContainerDied","Data":"9acfd064c597d58a07ab6ae946d10c18ba28df2e6231f44f9198ba5ab6f150ed"} Sep 11 12:06:50.579507 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.579394 2566 scope.go:117] "RemoveContainer" containerID="21181a6c7d819c51bc5f3f76894f876fbf5eeb67818119b4ec73fa2d45e46271" Sep 11 12:06:50.580752 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.580716 2566 generic.go:356] "Generic (PLEG): container finished" podID="58351aff-8939-4449-9523-e07001ad0840" containerID="cc54ae46fd72abf3a4fa7bd4009de713a9c29d58a4dabe56063feefce6b03655" exitCode=0 Sep 11 12:06:50.580844 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.580788 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9" event={"ID":"58351aff-8939-4449-9523-e07001ad0840","Type":"ContainerDied","Data":"cc54ae46fd72abf3a4fa7bd4009de713a9c29d58a4dabe56063feefce6b03655"} Sep 11 12:06:50.580844 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.580820 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9" Sep 11 12:06:50.580844 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.580832 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9" event={"ID":"58351aff-8939-4449-9523-e07001ad0840","Type":"ContainerDied","Data":"293b205b59864b8cd4100bb65ebb0c68a2b0441558aed22b0a4dbfb51f0c2422"} Sep 11 12:06:50.589518 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.589498 2566 scope.go:117] "RemoveContainer" containerID="21181a6c7d819c51bc5f3f76894f876fbf5eeb67818119b4ec73fa2d45e46271" Sep 11 12:06:50.589881 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:06:50.589839 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21181a6c7d819c51bc5f3f76894f876fbf5eeb67818119b4ec73fa2d45e46271\": container with ID starting with 21181a6c7d819c51bc5f3f76894f876fbf5eeb67818119b4ec73fa2d45e46271 not found: ID does not exist" containerID="21181a6c7d819c51bc5f3f76894f876fbf5eeb67818119b4ec73fa2d45e46271" Sep 11 12:06:50.589963 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.589891 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21181a6c7d819c51bc5f3f76894f876fbf5eeb67818119b4ec73fa2d45e46271"} err="failed to get container status \"21181a6c7d819c51bc5f3f76894f876fbf5eeb67818119b4ec73fa2d45e46271\": rpc error: code = NotFound desc = could not find container \"21181a6c7d819c51bc5f3f76894f876fbf5eeb67818119b4ec73fa2d45e46271\": container with ID starting with 21181a6c7d819c51bc5f3f76894f876fbf5eeb67818119b4ec73fa2d45e46271 not found: ID does not exist" Sep 11 12:06:50.589963 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.589909 2566 scope.go:117] "RemoveContainer" containerID="cc54ae46fd72abf3a4fa7bd4009de713a9c29d58a4dabe56063feefce6b03655" Sep 11 12:06:50.598136 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.598120 2566 scope.go:117] "RemoveContainer" containerID="cc54ae46fd72abf3a4fa7bd4009de713a9c29d58a4dabe56063feefce6b03655" Sep 11 12:06:50.598438 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:06:50.598417 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc54ae46fd72abf3a4fa7bd4009de713a9c29d58a4dabe56063feefce6b03655\": container with ID starting with cc54ae46fd72abf3a4fa7bd4009de713a9c29d58a4dabe56063feefce6b03655 not found: ID does not exist" containerID="cc54ae46fd72abf3a4fa7bd4009de713a9c29d58a4dabe56063feefce6b03655" Sep 11 12:06:50.598502 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.598447 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc54ae46fd72abf3a4fa7bd4009de713a9c29d58a4dabe56063feefce6b03655"} err="failed to get container status \"cc54ae46fd72abf3a4fa7bd4009de713a9c29d58a4dabe56063feefce6b03655\": rpc error: code = NotFound desc = could not find container \"cc54ae46fd72abf3a4fa7bd4009de713a9c29d58a4dabe56063feefce6b03655\": container with ID starting with cc54ae46fd72abf3a4fa7bd4009de713a9c29d58a4dabe56063feefce6b03655 not found: ID does not exist" Sep 11 12:06:50.629134 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.629094 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-cbc380-7b4d557d8b-52rrl"] Sep 11 12:06:50.643827 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.643794 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-cbc380-7b4d557d8b-52rrl"] Sep 11 12:06:50.661709 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.661675 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8fcpb\" (UniqueName: \"kubernetes.io/projected/431e293a-0b82-4210-aa0d-e7ed0bab4d00-kube-api-access-8fcpb\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:06:50.661709 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.661703 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/431e293a-0b82-4210-aa0d-e7ed0bab4d00-plugins-config-volume\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:06:50.665818 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.665790 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9"] Sep 11 12:06:50.679540 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.679512 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-cbc380-6484cbd685-8rqj9"] Sep 11 12:06:50.758739 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.758704 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="431e293a-0b82-4210-aa0d-e7ed0bab4d00" path="/var/lib/kubelet/pods/431e293a-0b82-4210-aa0d-e7ed0bab4d00/volumes" Sep 11 12:06:50.759146 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:50.759132 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58351aff-8939-4449-9523-e07001ad0840" path="/var/lib/kubelet/pods/58351aff-8939-4449-9523-e07001ad0840/volumes" Sep 11 12:06:51.226799 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:51.226754 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-24cfea9d-5dff57b685-lvzz4"] Sep 11 12:06:51.227293 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:51.227232 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcf12e46-656a-4d89-962a-b1a6cefaf643" containerName="istio-proxy" Sep 11 12:06:51.227293 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:51.227253 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcf12e46-656a-4d89-962a-b1a6cefaf643" containerName="istio-proxy" Sep 11 12:06:51.227293 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:51.227268 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="431e293a-0b82-4210-aa0d-e7ed0bab4d00" containerName="payload-pre-processing" Sep 11 12:06:51.227293 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:51.227277 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="431e293a-0b82-4210-aa0d-e7ed0bab4d00" containerName="payload-pre-processing" Sep 11 12:06:51.227566 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:51.227310 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58351aff-8939-4449-9523-e07001ad0840" containerName="maas-api" Sep 11 12:06:51.227566 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:51.227319 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="58351aff-8939-4449-9523-e07001ad0840" containerName="maas-api" Sep 11 12:06:51.227566 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:51.227331 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="332c2183-9cf2-4685-a2c2-401af97223a6" containerName="payload-processing" Sep 11 12:06:51.227566 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:51.227339 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="332c2183-9cf2-4685-a2c2-401af97223a6" containerName="payload-processing" Sep 11 12:06:51.227566 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:51.227428 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="58351aff-8939-4449-9523-e07001ad0840" containerName="maas-api" Sep 11 12:06:51.227566 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:51.227441 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="431e293a-0b82-4210-aa0d-e7ed0bab4d00" containerName="payload-pre-processing" Sep 11 12:06:51.227566 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:51.227451 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d23b88e-3a6a-472e-ab1e-904ca680e5f4" containerName="istio-proxy" Sep 11 12:06:51.227566 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:51.227460 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcf12e46-656a-4d89-962a-b1a6cefaf643" containerName="istio-proxy" Sep 11 12:06:51.227566 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:51.227469 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcf12e46-656a-4d89-962a-b1a6cefaf643" containerName="istio-proxy" Sep 11 12:06:51.227566 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:51.227477 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="332c2183-9cf2-4685-a2c2-401af97223a6" containerName="payload-processing" Sep 11 12:06:51.230301 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:51.230276 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-24cfea9d-5dff57b685-lvzz4" Sep 11 12:06:51.233093 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:51.233074 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-24cfea9d\"" Sep 11 12:06:51.246585 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:51.246554 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-24cfea9d-5dff57b685-lvzz4"] Sep 11 12:06:51.266158 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:51.266126 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/340dddb2-e23e-4a04-b274-efcd007be4b2-plugins-config-volume\") pod \"payload-processing-e2e-mt-24cfea9d-5dff57b685-lvzz4\" (UID: \"340dddb2-e23e-4a04-b274-efcd007be4b2\") " pod="openshift-ingress/payload-processing-e2e-mt-24cfea9d-5dff57b685-lvzz4" Sep 11 12:06:51.266415 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:51.266195 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xs27\" (UniqueName: \"kubernetes.io/projected/340dddb2-e23e-4a04-b274-efcd007be4b2-kube-api-access-2xs27\") pod \"payload-processing-e2e-mt-24cfea9d-5dff57b685-lvzz4\" (UID: \"340dddb2-e23e-4a04-b274-efcd007be4b2\") " pod="openshift-ingress/payload-processing-e2e-mt-24cfea9d-5dff57b685-lvzz4" Sep 11 12:06:51.367274 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:51.367181 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/340dddb2-e23e-4a04-b274-efcd007be4b2-plugins-config-volume\") pod \"payload-processing-e2e-mt-24cfea9d-5dff57b685-lvzz4\" (UID: \"340dddb2-e23e-4a04-b274-efcd007be4b2\") " pod="openshift-ingress/payload-processing-e2e-mt-24cfea9d-5dff57b685-lvzz4" Sep 11 12:06:51.367458 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:51.367333 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xs27\" (UniqueName: \"kubernetes.io/projected/340dddb2-e23e-4a04-b274-efcd007be4b2-kube-api-access-2xs27\") pod \"payload-processing-e2e-mt-24cfea9d-5dff57b685-lvzz4\" (UID: \"340dddb2-e23e-4a04-b274-efcd007be4b2\") " pod="openshift-ingress/payload-processing-e2e-mt-24cfea9d-5dff57b685-lvzz4" Sep 11 12:06:51.368084 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:51.368057 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/340dddb2-e23e-4a04-b274-efcd007be4b2-plugins-config-volume\") pod \"payload-processing-e2e-mt-24cfea9d-5dff57b685-lvzz4\" (UID: \"340dddb2-e23e-4a04-b274-efcd007be4b2\") " pod="openshift-ingress/payload-processing-e2e-mt-24cfea9d-5dff57b685-lvzz4" Sep 11 12:06:51.379276 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:51.379249 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xs27\" (UniqueName: \"kubernetes.io/projected/340dddb2-e23e-4a04-b274-efcd007be4b2-kube-api-access-2xs27\") pod \"payload-processing-e2e-mt-24cfea9d-5dff57b685-lvzz4\" (UID: \"340dddb2-e23e-4a04-b274-efcd007be4b2\") " pod="openshift-ingress/payload-processing-e2e-mt-24cfea9d-5dff57b685-lvzz4" Sep 11 12:06:51.540565 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:51.540527 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-24cfea9d-5dff57b685-lvzz4" Sep 11 12:06:51.812407 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:51.812331 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-24cfea9d-5dff57b685-lvzz4"] Sep 11 12:06:51.814668 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:06:51.814639 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod340dddb2_e23e_4a04_b274_efcd007be4b2.slice/crio-68945c1ae90162b4c20dfc0acc410e42adb8dce92aede13a4615455c4e383299 WatchSource:0}: Error finding container 68945c1ae90162b4c20dfc0acc410e42adb8dce92aede13a4615455c4e383299: Status 404 returned error can't find the container with id 68945c1ae90162b4c20dfc0acc410e42adb8dce92aede13a4615455c4e383299 Sep 11 12:06:52.595487 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:52.595450 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-24cfea9d-5dff57b685-lvzz4" event={"ID":"340dddb2-e23e-4a04-b274-efcd007be4b2","Type":"ContainerStarted","Data":"30dcb1159e8def94967a0643218a6734a1438e707b88c443cfd10a457cbd86dc"} Sep 11 12:06:52.595911 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:52.595506 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-24cfea9d-5dff57b685-lvzz4" event={"ID":"340dddb2-e23e-4a04-b274-efcd007be4b2","Type":"ContainerStarted","Data":"68945c1ae90162b4c20dfc0acc410e42adb8dce92aede13a4615455c4e383299"} Sep 11 12:06:52.595911 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:52.595560 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-24cfea9d-5dff57b685-lvzz4" Sep 11 12:06:52.641041 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:52.640987 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-24cfea9d-5dff57b685-lvzz4" podStartSLOduration=1.6409691789999998 podStartE2EDuration="1.640969179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:06:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:06:52.638334684 +0000 UTC m=+1144.403307860" watchObservedRunningTime="2026-09-11 12:06:52.640969179 +0000 UTC m=+1144.405942355" Sep 11 12:06:57.268585 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:57.268550 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2"] Sep 11 12:06:57.269094 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:06:57.268816 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" podUID="67bee4aa-84b4-4138-9ecd-51edc10f2e1a" containerName="istio-proxy" containerID="cri-o://c0c71cb28d33a6c28b1a61a8e64bf7a9d2a8b27974547fa422d7ed34af415c94" gracePeriod=30 Sep 11 12:07:02.511753 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.511726 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" Sep 11 12:07:02.570701 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.570672 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-istiod-ca-cert\") pod \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\" (UID: \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\") " Sep 11 12:07:02.570883 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.570748 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-workload-socket\") pod \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\" (UID: \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\") " Sep 11 12:07:02.570883 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.570770 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-workload-certs\") pod \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\" (UID: \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\") " Sep 11 12:07:02.570883 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.570794 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-credential-socket\") pod \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\" (UID: \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\") " Sep 11 12:07:02.570883 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.570825 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-istio-podinfo\") pod \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\" (UID: \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\") " Sep 11 12:07:02.570883 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.570852 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vbp6v\" (UniqueName: \"kubernetes.io/projected/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-kube-api-access-vbp6v\") pod \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\" (UID: \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\") " Sep 11 12:07:02.571141 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.571020 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-istio-data\") pod \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\" (UID: \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\") " Sep 11 12:07:02.571141 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.571060 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-istio-token\") pod \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\" (UID: \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\") " Sep 11 12:07:02.571141 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.571127 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-istio-envoy\") pod \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\" (UID: \"67bee4aa-84b4-4138-9ecd-51edc10f2e1a\") " Sep 11 12:07:02.571293 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.571109 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "67bee4aa-84b4-4138-9ecd-51edc10f2e1a" (UID: "67bee4aa-84b4-4138-9ecd-51edc10f2e1a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 12:07:02.571293 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.571139 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "67bee4aa-84b4-4138-9ecd-51edc10f2e1a" (UID: "67bee4aa-84b4-4138-9ecd-51edc10f2e1a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:07:02.571293 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.571157 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "67bee4aa-84b4-4138-9ecd-51edc10f2e1a" (UID: "67bee4aa-84b4-4138-9ecd-51edc10f2e1a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:07:02.571430 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.571284 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-istio-data" (OuterVolumeSpecName: "istio-data") pod "67bee4aa-84b4-4138-9ecd-51edc10f2e1a" (UID: "67bee4aa-84b4-4138-9ecd-51edc10f2e1a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:07:02.571430 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.571390 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "67bee4aa-84b4-4138-9ecd-51edc10f2e1a" (UID: "67bee4aa-84b4-4138-9ecd-51edc10f2e1a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:07:02.571541 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.571448 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-istio-data\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:07:02.571541 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.571468 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-istiod-ca-cert\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:07:02.571541 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.571481 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-workload-socket\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:07:02.571541 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.571496 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-workload-certs\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:07:02.573239 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.573194 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "67bee4aa-84b4-4138-9ecd-51edc10f2e1a" (UID: "67bee4aa-84b4-4138-9ecd-51edc10f2e1a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 11 12:07:02.573343 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.573249 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "67bee4aa-84b4-4138-9ecd-51edc10f2e1a" (UID: "67bee4aa-84b4-4138-9ecd-51edc10f2e1a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:07:02.573343 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.573286 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-kube-api-access-vbp6v" (OuterVolumeSpecName: "kube-api-access-vbp6v") pod "67bee4aa-84b4-4138-9ecd-51edc10f2e1a" (UID: "67bee4aa-84b4-4138-9ecd-51edc10f2e1a"). InnerVolumeSpecName "kube-api-access-vbp6v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:07:02.573343 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.573307 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-istio-token" (OuterVolumeSpecName: "istio-token") pod "67bee4aa-84b4-4138-9ecd-51edc10f2e1a" (UID: "67bee4aa-84b4-4138-9ecd-51edc10f2e1a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:07:02.646714 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.646677 2566 generic.go:356] "Generic (PLEG): container finished" podID="67bee4aa-84b4-4138-9ecd-51edc10f2e1a" containerID="c0c71cb28d33a6c28b1a61a8e64bf7a9d2a8b27974547fa422d7ed34af415c94" exitCode=0 Sep 11 12:07:02.646881 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.646720 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" Sep 11 12:07:02.646881 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.646753 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" event={"ID":"67bee4aa-84b4-4138-9ecd-51edc10f2e1a","Type":"ContainerDied","Data":"c0c71cb28d33a6c28b1a61a8e64bf7a9d2a8b27974547fa422d7ed34af415c94"} Sep 11 12:07:02.646881 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.646809 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2" event={"ID":"67bee4aa-84b4-4138-9ecd-51edc10f2e1a","Type":"ContainerDied","Data":"d7a41fc4156ed956460359dff7a9f08f7fd8a0137462f044e37ce088309baed7"} Sep 11 12:07:02.646881 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.646832 2566 scope.go:117] "RemoveContainer" containerID="c0c71cb28d33a6c28b1a61a8e64bf7a9d2a8b27974547fa422d7ed34af415c94" Sep 11 12:07:02.655968 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.655948 2566 scope.go:117] "RemoveContainer" containerID="abf3978897df7df56370bfba52738804adc222b5f91c8eaa1d9fb20b24575342" Sep 11 12:07:02.663954 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.663933 2566 scope.go:117] "RemoveContainer" containerID="c0c71cb28d33a6c28b1a61a8e64bf7a9d2a8b27974547fa422d7ed34af415c94" Sep 11 12:07:02.664218 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:07:02.664198 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0c71cb28d33a6c28b1a61a8e64bf7a9d2a8b27974547fa422d7ed34af415c94\": container with ID starting with c0c71cb28d33a6c28b1a61a8e64bf7a9d2a8b27974547fa422d7ed34af415c94 not found: ID does not exist" containerID="c0c71cb28d33a6c28b1a61a8e64bf7a9d2a8b27974547fa422d7ed34af415c94" Sep 11 12:07:02.664297 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.664230 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0c71cb28d33a6c28b1a61a8e64bf7a9d2a8b27974547fa422d7ed34af415c94"} err="failed to get container status \"c0c71cb28d33a6c28b1a61a8e64bf7a9d2a8b27974547fa422d7ed34af415c94\": rpc error: code = NotFound desc = could not find container \"c0c71cb28d33a6c28b1a61a8e64bf7a9d2a8b27974547fa422d7ed34af415c94\": container with ID starting with c0c71cb28d33a6c28b1a61a8e64bf7a9d2a8b27974547fa422d7ed34af415c94 not found: ID does not exist" Sep 11 12:07:02.664297 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.664255 2566 scope.go:117] "RemoveContainer" containerID="abf3978897df7df56370bfba52738804adc222b5f91c8eaa1d9fb20b24575342" Sep 11 12:07:02.664498 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:07:02.664481 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abf3978897df7df56370bfba52738804adc222b5f91c8eaa1d9fb20b24575342\": container with ID starting with abf3978897df7df56370bfba52738804adc222b5f91c8eaa1d9fb20b24575342 not found: ID does not exist" containerID="abf3978897df7df56370bfba52738804adc222b5f91c8eaa1d9fb20b24575342" Sep 11 12:07:02.664536 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.664504 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abf3978897df7df56370bfba52738804adc222b5f91c8eaa1d9fb20b24575342"} err="failed to get container status \"abf3978897df7df56370bfba52738804adc222b5f91c8eaa1d9fb20b24575342\": rpc error: code = NotFound desc = could not find container \"abf3978897df7df56370bfba52738804adc222b5f91c8eaa1d9fb20b24575342\": container with ID starting with abf3978897df7df56370bfba52738804adc222b5f91c8eaa1d9fb20b24575342 not found: ID does not exist" Sep 11 12:07:02.672222 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.672187 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-istio-token\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:07:02.672222 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.672219 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-istio-envoy\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:07:02.672355 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.672232 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-credential-socket\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:07:02.672355 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.672245 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-istio-podinfo\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:07:02.672355 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.672258 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbp6v\" (UniqueName: \"kubernetes.io/projected/67bee4aa-84b4-4138-9ecd-51edc10f2e1a-kube-api-access-vbp6v\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:07:02.673609 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.673588 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2"] Sep 11 12:07:02.681018 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.680995 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-cbc380-openshift-default-74d656dc84-hcxg2"] Sep 11 12:07:02.758531 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:02.758498 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67bee4aa-84b4-4138-9ecd-51edc10f2e1a" path="/var/lib/kubelet/pods/67bee4aa-84b4-4138-9ecd-51edc10f2e1a/volumes" Sep 11 12:07:03.603768 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:03.603736 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-24cfea9d-5dff57b685-lvzz4" Sep 11 12:07:08.275950 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:08.275906 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-12d2a9-5944db5cd9-4wrcl"] Sep 11 12:07:08.276337 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:08.276146 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-12d2a9-5944db5cd9-4wrcl" podUID="efa26d3d-e23b-4e61-a9f6-fe0815ceef93" containerName="payload-processing" containerID="cri-o://3c4ebd19b0c03f7c86c9ff429547c880df6d74c13d392c0255abcd77b4413dfa" gracePeriod=30 Sep 11 12:07:08.518036 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:08.518013 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-12d2a9-5944db5cd9-4wrcl" Sep 11 12:07:08.622011 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:08.621976 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efa26d3d-e23b-4e61-a9f6-fe0815ceef93-plugins-config-volume\") pod \"efa26d3d-e23b-4e61-a9f6-fe0815ceef93\" (UID: \"efa26d3d-e23b-4e61-a9f6-fe0815ceef93\") " Sep 11 12:07:08.622217 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:08.622080 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rmphx\" (UniqueName: \"kubernetes.io/projected/efa26d3d-e23b-4e61-a9f6-fe0815ceef93-kube-api-access-rmphx\") pod \"efa26d3d-e23b-4e61-a9f6-fe0815ceef93\" (UID: \"efa26d3d-e23b-4e61-a9f6-fe0815ceef93\") " Sep 11 12:07:08.622404 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:08.622377 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efa26d3d-e23b-4e61-a9f6-fe0815ceef93-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "efa26d3d-e23b-4e61-a9f6-fe0815ceef93" (UID: "efa26d3d-e23b-4e61-a9f6-fe0815ceef93"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 12:07:08.624150 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:08.624120 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efa26d3d-e23b-4e61-a9f6-fe0815ceef93-kube-api-access-rmphx" (OuterVolumeSpecName: "kube-api-access-rmphx") pod "efa26d3d-e23b-4e61-a9f6-fe0815ceef93" (UID: "efa26d3d-e23b-4e61-a9f6-fe0815ceef93"). InnerVolumeSpecName "kube-api-access-rmphx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:07:08.674924 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:08.674896 2566 generic.go:356] "Generic (PLEG): container finished" podID="efa26d3d-e23b-4e61-a9f6-fe0815ceef93" containerID="3c4ebd19b0c03f7c86c9ff429547c880df6d74c13d392c0255abcd77b4413dfa" exitCode=0 Sep 11 12:07:08.675037 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:08.674941 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-12d2a9-5944db5cd9-4wrcl" event={"ID":"efa26d3d-e23b-4e61-a9f6-fe0815ceef93","Type":"ContainerDied","Data":"3c4ebd19b0c03f7c86c9ff429547c880df6d74c13d392c0255abcd77b4413dfa"} Sep 11 12:07:08.675037 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:08.674947 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-12d2a9-5944db5cd9-4wrcl" Sep 11 12:07:08.675037 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:08.674984 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-12d2a9-5944db5cd9-4wrcl" event={"ID":"efa26d3d-e23b-4e61-a9f6-fe0815ceef93","Type":"ContainerDied","Data":"e1ed3e465ebf9d00725257bfeac6c4dd588dba4bb88ed7bff0fdca01b12c24a2"} Sep 11 12:07:08.675037 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:08.675005 2566 scope.go:117] "RemoveContainer" containerID="3c4ebd19b0c03f7c86c9ff429547c880df6d74c13d392c0255abcd77b4413dfa" Sep 11 12:07:08.683541 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:08.683526 2566 scope.go:117] "RemoveContainer" containerID="3c4ebd19b0c03f7c86c9ff429547c880df6d74c13d392c0255abcd77b4413dfa" Sep 11 12:07:08.683826 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:07:08.683809 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c4ebd19b0c03f7c86c9ff429547c880df6d74c13d392c0255abcd77b4413dfa\": container with ID starting with 3c4ebd19b0c03f7c86c9ff429547c880df6d74c13d392c0255abcd77b4413dfa not found: ID does not exist" containerID="3c4ebd19b0c03f7c86c9ff429547c880df6d74c13d392c0255abcd77b4413dfa" Sep 11 12:07:08.683887 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:08.683836 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c4ebd19b0c03f7c86c9ff429547c880df6d74c13d392c0255abcd77b4413dfa"} err="failed to get container status \"3c4ebd19b0c03f7c86c9ff429547c880df6d74c13d392c0255abcd77b4413dfa\": rpc error: code = NotFound desc = could not find container \"3c4ebd19b0c03f7c86c9ff429547c880df6d74c13d392c0255abcd77b4413dfa\": container with ID starting with 3c4ebd19b0c03f7c86c9ff429547c880df6d74c13d392c0255abcd77b4413dfa not found: ID does not exist" Sep 11 12:07:08.701885 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:08.701842 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-12d2a9-5944db5cd9-4wrcl"] Sep 11 12:07:08.703554 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:08.703530 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-12d2a9-5944db5cd9-4wrcl"] Sep 11 12:07:08.723147 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:08.723124 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmphx\" (UniqueName: \"kubernetes.io/projected/efa26d3d-e23b-4e61-a9f6-fe0815ceef93-kube-api-access-rmphx\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:07:08.723147 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:08.723145 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efa26d3d-e23b-4e61-a9f6-fe0815ceef93-plugins-config-volume\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:07:08.757670 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:08.757650 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efa26d3d-e23b-4e61-a9f6-fe0815ceef93" path="/var/lib/kubelet/pods/efa26d3d-e23b-4e61-a9f6-fe0815ceef93/volumes" Sep 11 12:07:17.605672 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.605624 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz"] Sep 11 12:07:17.606172 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.606097 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67bee4aa-84b4-4138-9ecd-51edc10f2e1a" containerName="istio-proxy" Sep 11 12:07:17.606172 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.606117 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="67bee4aa-84b4-4138-9ecd-51edc10f2e1a" containerName="istio-proxy" Sep 11 12:07:17.606172 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.606138 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67bee4aa-84b4-4138-9ecd-51edc10f2e1a" containerName="istio-proxy" Sep 11 12:07:17.606172 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.606147 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="67bee4aa-84b4-4138-9ecd-51edc10f2e1a" containerName="istio-proxy" Sep 11 12:07:17.606172 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.606158 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efa26d3d-e23b-4e61-a9f6-fe0815ceef93" containerName="payload-processing" Sep 11 12:07:17.606172 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.606167 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="efa26d3d-e23b-4e61-a9f6-fe0815ceef93" containerName="payload-processing" Sep 11 12:07:17.606466 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.606189 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcf12e46-656a-4d89-962a-b1a6cefaf643" containerName="istio-proxy" Sep 11 12:07:17.606466 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.606197 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcf12e46-656a-4d89-962a-b1a6cefaf643" containerName="istio-proxy" Sep 11 12:07:17.606466 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.606270 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="efa26d3d-e23b-4e61-a9f6-fe0815ceef93" containerName="payload-processing" Sep 11 12:07:17.606466 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.606282 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="67bee4aa-84b4-4138-9ecd-51edc10f2e1a" containerName="istio-proxy" Sep 11 12:07:17.606466 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.606292 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="67bee4aa-84b4-4138-9ecd-51edc10f2e1a" containerName="istio-proxy" Sep 11 12:07:17.608746 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.608715 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" Sep 11 12:07:17.611288 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.611265 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-af081c-openshift-default-dockercfg-clbtw\"" Sep 11 12:07:17.625295 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.625270 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz"] Sep 11 12:07:17.705565 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.705525 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcvsd\" (UniqueName: \"kubernetes.io/projected/47192df9-4559-4286-a94e-1f01440c4c65-kube-api-access-qcvsd\") pod \"e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz\" (UID: \"47192df9-4559-4286-a94e-1f01440c4c65\") " pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" Sep 11 12:07:17.705752 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.705583 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47192df9-4559-4286-a94e-1f01440c4c65-istiod-ca-cert\") pod \"e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz\" (UID: \"47192df9-4559-4286-a94e-1f01440c4c65\") " pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" Sep 11 12:07:17.705752 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.705661 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47192df9-4559-4286-a94e-1f01440c4c65-istio-data\") pod \"e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz\" (UID: \"47192df9-4559-4286-a94e-1f01440c4c65\") " pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" Sep 11 12:07:17.705752 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.705742 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47192df9-4559-4286-a94e-1f01440c4c65-istio-envoy\") pod \"e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz\" (UID: \"47192df9-4559-4286-a94e-1f01440c4c65\") " pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" Sep 11 12:07:17.705876 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.705786 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47192df9-4559-4286-a94e-1f01440c4c65-workload-socket\") pod \"e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz\" (UID: \"47192df9-4559-4286-a94e-1f01440c4c65\") " pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" Sep 11 12:07:17.705876 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.705815 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47192df9-4559-4286-a94e-1f01440c4c65-istio-token\") pod \"e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz\" (UID: \"47192df9-4559-4286-a94e-1f01440c4c65\") " pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" Sep 11 12:07:17.705876 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.705842 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47192df9-4559-4286-a94e-1f01440c4c65-workload-certs\") pod \"e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz\" (UID: \"47192df9-4559-4286-a94e-1f01440c4c65\") " pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" Sep 11 12:07:17.705986 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.705920 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47192df9-4559-4286-a94e-1f01440c4c65-credential-socket\") pod \"e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz\" (UID: \"47192df9-4559-4286-a94e-1f01440c4c65\") " pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" Sep 11 12:07:17.705986 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.705946 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47192df9-4559-4286-a94e-1f01440c4c65-istio-podinfo\") pod \"e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz\" (UID: \"47192df9-4559-4286-a94e-1f01440c4c65\") " pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" Sep 11 12:07:17.807142 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.807101 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47192df9-4559-4286-a94e-1f01440c4c65-istio-envoy\") pod \"e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz\" (UID: \"47192df9-4559-4286-a94e-1f01440c4c65\") " pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" Sep 11 12:07:17.807142 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.807145 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47192df9-4559-4286-a94e-1f01440c4c65-workload-socket\") pod \"e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz\" (UID: \"47192df9-4559-4286-a94e-1f01440c4c65\") " pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" Sep 11 12:07:17.807384 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.807162 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47192df9-4559-4286-a94e-1f01440c4c65-istio-token\") pod \"e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz\" (UID: \"47192df9-4559-4286-a94e-1f01440c4c65\") " pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" Sep 11 12:07:17.807384 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.807186 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47192df9-4559-4286-a94e-1f01440c4c65-workload-certs\") pod \"e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz\" (UID: \"47192df9-4559-4286-a94e-1f01440c4c65\") " pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" Sep 11 12:07:17.807384 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.807231 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47192df9-4559-4286-a94e-1f01440c4c65-credential-socket\") pod \"e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz\" (UID: \"47192df9-4559-4286-a94e-1f01440c4c65\") " pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" Sep 11 12:07:17.807384 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.807255 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47192df9-4559-4286-a94e-1f01440c4c65-istio-podinfo\") pod \"e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz\" (UID: \"47192df9-4559-4286-a94e-1f01440c4c65\") " pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" Sep 11 12:07:17.807384 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.807304 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcvsd\" (UniqueName: \"kubernetes.io/projected/47192df9-4559-4286-a94e-1f01440c4c65-kube-api-access-qcvsd\") pod \"e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz\" (UID: \"47192df9-4559-4286-a94e-1f01440c4c65\") " pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" Sep 11 12:07:17.807384 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.807328 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47192df9-4559-4286-a94e-1f01440c4c65-istiod-ca-cert\") pod \"e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz\" (UID: \"47192df9-4559-4286-a94e-1f01440c4c65\") " pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" Sep 11 12:07:17.807384 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.807377 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47192df9-4559-4286-a94e-1f01440c4c65-istio-data\") pod \"e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz\" (UID: \"47192df9-4559-4286-a94e-1f01440c4c65\") " pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" Sep 11 12:07:17.807801 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.807637 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47192df9-4559-4286-a94e-1f01440c4c65-credential-socket\") pod \"e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz\" (UID: \"47192df9-4559-4286-a94e-1f01440c4c65\") " pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" Sep 11 12:07:17.807801 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.807648 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47192df9-4559-4286-a94e-1f01440c4c65-workload-socket\") pod \"e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz\" (UID: \"47192df9-4559-4286-a94e-1f01440c4c65\") " pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" Sep 11 12:07:17.807801 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.807706 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47192df9-4559-4286-a94e-1f01440c4c65-workload-certs\") pod \"e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz\" (UID: \"47192df9-4559-4286-a94e-1f01440c4c65\") " pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" Sep 11 12:07:17.807993 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.807972 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47192df9-4559-4286-a94e-1f01440c4c65-istio-data\") pod \"e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz\" (UID: \"47192df9-4559-4286-a94e-1f01440c4c65\") " pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" Sep 11 12:07:17.808160 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.808142 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47192df9-4559-4286-a94e-1f01440c4c65-istiod-ca-cert\") pod \"e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz\" (UID: \"47192df9-4559-4286-a94e-1f01440c4c65\") " pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" Sep 11 12:07:17.809592 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.809562 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47192df9-4559-4286-a94e-1f01440c4c65-istio-envoy\") pod \"e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz\" (UID: \"47192df9-4559-4286-a94e-1f01440c4c65\") " pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" Sep 11 12:07:17.810077 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.810057 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47192df9-4559-4286-a94e-1f01440c4c65-istio-podinfo\") pod \"e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz\" (UID: \"47192df9-4559-4286-a94e-1f01440c4c65\") " pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" Sep 11 12:07:17.816359 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.816329 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcvsd\" (UniqueName: \"kubernetes.io/projected/47192df9-4559-4286-a94e-1f01440c4c65-kube-api-access-qcvsd\") pod \"e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz\" (UID: \"47192df9-4559-4286-a94e-1f01440c4c65\") " pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" Sep 11 12:07:17.816584 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.816563 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47192df9-4559-4286-a94e-1f01440c4c65-istio-token\") pod \"e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz\" (UID: \"47192df9-4559-4286-a94e-1f01440c4c65\") " pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" Sep 11 12:07:17.918815 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:17.918716 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" Sep 11 12:07:18.198228 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:18.198199 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz"] Sep 11 12:07:18.200699 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:07:18.200655 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47192df9_4559_4286_a94e_1f01440c4c65.slice/crio-98b6c7d39e30ff1e372b2fc9c4689b6b498bb8b8e6354ae7c2c7a78a9bf787dc WatchSource:0}: Error finding container 98b6c7d39e30ff1e372b2fc9c4689b6b498bb8b8e6354ae7c2c7a78a9bf787dc: Status 404 returned error can't find the container with id 98b6c7d39e30ff1e372b2fc9c4689b6b498bb8b8e6354ae7c2c7a78a9bf787dc Sep 11 12:07:18.203142 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:18.203107 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:07:18.203227 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:18.203175 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:07:18.203227 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:18.203206 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:07:18.723880 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:18.723838 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" event={"ID":"47192df9-4559-4286-a94e-1f01440c4c65","Type":"ContainerStarted","Data":"f2ad890a0a45f606d2171a404cd28259f497956d3dab85c93803c892682aaa90"} Sep 11 12:07:18.724252 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:18.723901 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" event={"ID":"47192df9-4559-4286-a94e-1f01440c4c65","Type":"ContainerStarted","Data":"98b6c7d39e30ff1e372b2fc9c4689b6b498bb8b8e6354ae7c2c7a78a9bf787dc"} Sep 11 12:07:18.749803 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:18.749740 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" podStartSLOduration=1.749720533 podStartE2EDuration="1.749720533s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:07:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:07:18.747965808 +0000 UTC m=+1170.512938982" watchObservedRunningTime="2026-09-11 12:07:18.749720533 +0000 UTC m=+1170.514693709" Sep 11 12:07:18.919703 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:18.919668 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" Sep 11 12:07:19.925229 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:19.925190 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" Sep 11 12:07:20.733661 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:20.733629 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" Sep 11 12:07:20.734556 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:20.734534 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" Sep 11 12:07:22.639033 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:22.638999 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-af081c-74b8976d5c-tvm6b"] Sep 11 12:07:22.641639 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:22.641615 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-af081c-74b8976d5c-tvm6b" Sep 11 12:07:22.644782 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:22.644758 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-af081c\"" Sep 11 12:07:22.645647 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:22.645628 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-af081c-dockercfg-7hvkm\"" Sep 11 12:07:22.652172 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:22.652139 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-af081c-74b8976d5c-tvm6b"] Sep 11 12:07:22.751563 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:22.751529 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96a0d2ef-39dd-476e-a527-53e9af9cca98-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-af081c-74b8976d5c-tvm6b\" (UID: \"96a0d2ef-39dd-476e-a527-53e9af9cca98\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-af081c-74b8976d5c-tvm6b" Sep 11 12:07:22.751563 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:22.751569 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqfvc\" (UniqueName: \"kubernetes.io/projected/96a0d2ef-39dd-476e-a527-53e9af9cca98-kube-api-access-wqfvc\") pod \"payload-processing-e2e-shared-a-w3-af081c-74b8976d5c-tvm6b\" (UID: \"96a0d2ef-39dd-476e-a527-53e9af9cca98\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-af081c-74b8976d5c-tvm6b" Sep 11 12:07:22.853010 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:22.852971 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96a0d2ef-39dd-476e-a527-53e9af9cca98-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-af081c-74b8976d5c-tvm6b\" (UID: \"96a0d2ef-39dd-476e-a527-53e9af9cca98\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-af081c-74b8976d5c-tvm6b" Sep 11 12:07:22.853010 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:22.853014 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqfvc\" (UniqueName: \"kubernetes.io/projected/96a0d2ef-39dd-476e-a527-53e9af9cca98-kube-api-access-wqfvc\") pod \"payload-processing-e2e-shared-a-w3-af081c-74b8976d5c-tvm6b\" (UID: \"96a0d2ef-39dd-476e-a527-53e9af9cca98\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-af081c-74b8976d5c-tvm6b" Sep 11 12:07:22.853749 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:22.853728 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96a0d2ef-39dd-476e-a527-53e9af9cca98-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-af081c-74b8976d5c-tvm6b\" (UID: \"96a0d2ef-39dd-476e-a527-53e9af9cca98\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-af081c-74b8976d5c-tvm6b" Sep 11 12:07:22.861747 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:22.861722 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqfvc\" (UniqueName: \"kubernetes.io/projected/96a0d2ef-39dd-476e-a527-53e9af9cca98-kube-api-access-wqfvc\") pod \"payload-processing-e2e-shared-a-w3-af081c-74b8976d5c-tvm6b\" (UID: \"96a0d2ef-39dd-476e-a527-53e9af9cca98\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-af081c-74b8976d5c-tvm6b" Sep 11 12:07:22.954166 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:22.954075 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-af081c-74b8976d5c-tvm6b" Sep 11 12:07:23.229257 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:23.229232 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-af081c-74b8976d5c-tvm6b"] Sep 11 12:07:23.232002 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:07:23.231973 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96a0d2ef_39dd_476e_a527_53e9af9cca98.slice/crio-9970ddee2ac26168766c0473d272a31e6f12f28ba644bca7f2909996fc740f0c WatchSource:0}: Error finding container 9970ddee2ac26168766c0473d272a31e6f12f28ba644bca7f2909996fc740f0c: Status 404 returned error can't find the container with id 9970ddee2ac26168766c0473d272a31e6f12f28ba644bca7f2909996fc740f0c Sep 11 12:07:23.750010 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:23.749972 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-af081c-74b8976d5c-tvm6b" event={"ID":"96a0d2ef-39dd-476e-a527-53e9af9cca98","Type":"ContainerStarted","Data":"f9e69622b81d1cd118d4e0bf38c64eecf6368e78f0228a9bbf3259522838d61a"} Sep 11 12:07:23.750491 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:23.750033 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-af081c-74b8976d5c-tvm6b" event={"ID":"96a0d2ef-39dd-476e-a527-53e9af9cca98","Type":"ContainerStarted","Data":"9970ddee2ac26168766c0473d272a31e6f12f28ba644bca7f2909996fc740f0c"} Sep 11 12:07:23.750491 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:23.750105 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-af081c-74b8976d5c-tvm6b" Sep 11 12:07:34.295739 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:34.295647 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-af081c-74b8976d5c-tvm6b" podStartSLOduration=12.295622878 podStartE2EDuration="12.295622878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:07:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:07:23.773988617 +0000 UTC m=+1175.538961797" watchObservedRunningTime="2026-09-11 12:07:34.295622878 +0000 UTC m=+1186.060596056" Sep 11 12:07:34.297930 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:34.297846 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-24cfea9d-5dff57b685-lvzz4"] Sep 11 12:07:34.298241 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:34.298123 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-24cfea9d-5dff57b685-lvzz4" podUID="340dddb2-e23e-4a04-b274-efcd007be4b2" containerName="payload-processing" containerID="cri-o://30dcb1159e8def94967a0643218a6734a1438e707b88c443cfd10a457cbd86dc" gracePeriod=30 Sep 11 12:07:34.562652 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:34.562622 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-24cfea9d-5dff57b685-lvzz4" Sep 11 12:07:34.658055 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:34.658023 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/340dddb2-e23e-4a04-b274-efcd007be4b2-plugins-config-volume\") pod \"340dddb2-e23e-4a04-b274-efcd007be4b2\" (UID: \"340dddb2-e23e-4a04-b274-efcd007be4b2\") " Sep 11 12:07:34.658246 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:34.658102 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2xs27\" (UniqueName: \"kubernetes.io/projected/340dddb2-e23e-4a04-b274-efcd007be4b2-kube-api-access-2xs27\") pod \"340dddb2-e23e-4a04-b274-efcd007be4b2\" (UID: \"340dddb2-e23e-4a04-b274-efcd007be4b2\") " Sep 11 12:07:34.658432 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:34.658407 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/340dddb2-e23e-4a04-b274-efcd007be4b2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "340dddb2-e23e-4a04-b274-efcd007be4b2" (UID: "340dddb2-e23e-4a04-b274-efcd007be4b2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 12:07:34.660269 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:34.660241 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/340dddb2-e23e-4a04-b274-efcd007be4b2-kube-api-access-2xs27" (OuterVolumeSpecName: "kube-api-access-2xs27") pod "340dddb2-e23e-4a04-b274-efcd007be4b2" (UID: "340dddb2-e23e-4a04-b274-efcd007be4b2"). InnerVolumeSpecName "kube-api-access-2xs27". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:07:34.758883 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:34.758834 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-af081c-74b8976d5c-tvm6b" Sep 11 12:07:34.759432 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:34.759414 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/340dddb2-e23e-4a04-b274-efcd007be4b2-plugins-config-volume\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:07:34.759550 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:34.759531 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2xs27\" (UniqueName: \"kubernetes.io/projected/340dddb2-e23e-4a04-b274-efcd007be4b2-kube-api-access-2xs27\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:07:34.801715 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:34.801618 2566 generic.go:356] "Generic (PLEG): container finished" podID="340dddb2-e23e-4a04-b274-efcd007be4b2" containerID="30dcb1159e8def94967a0643218a6734a1438e707b88c443cfd10a457cbd86dc" exitCode=0 Sep 11 12:07:34.801715 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:34.801663 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-24cfea9d-5dff57b685-lvzz4" Sep 11 12:07:34.801715 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:34.801697 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-24cfea9d-5dff57b685-lvzz4" event={"ID":"340dddb2-e23e-4a04-b274-efcd007be4b2","Type":"ContainerDied","Data":"30dcb1159e8def94967a0643218a6734a1438e707b88c443cfd10a457cbd86dc"} Sep 11 12:07:34.802061 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:34.801747 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-24cfea9d-5dff57b685-lvzz4" event={"ID":"340dddb2-e23e-4a04-b274-efcd007be4b2","Type":"ContainerDied","Data":"68945c1ae90162b4c20dfc0acc410e42adb8dce92aede13a4615455c4e383299"} Sep 11 12:07:34.802061 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:34.801770 2566 scope.go:117] "RemoveContainer" containerID="30dcb1159e8def94967a0643218a6734a1438e707b88c443cfd10a457cbd86dc" Sep 11 12:07:34.812342 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:34.812315 2566 scope.go:117] "RemoveContainer" containerID="30dcb1159e8def94967a0643218a6734a1438e707b88c443cfd10a457cbd86dc" Sep 11 12:07:34.812792 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:07:34.812765 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30dcb1159e8def94967a0643218a6734a1438e707b88c443cfd10a457cbd86dc\": container with ID starting with 30dcb1159e8def94967a0643218a6734a1438e707b88c443cfd10a457cbd86dc not found: ID does not exist" containerID="30dcb1159e8def94967a0643218a6734a1438e707b88c443cfd10a457cbd86dc" Sep 11 12:07:34.812927 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:34.812801 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30dcb1159e8def94967a0643218a6734a1438e707b88c443cfd10a457cbd86dc"} err="failed to get container status \"30dcb1159e8def94967a0643218a6734a1438e707b88c443cfd10a457cbd86dc\": rpc error: code = NotFound desc = could not find container \"30dcb1159e8def94967a0643218a6734a1438e707b88c443cfd10a457cbd86dc\": container with ID starting with 30dcb1159e8def94967a0643218a6734a1438e707b88c443cfd10a457cbd86dc not found: ID does not exist" Sep 11 12:07:34.832284 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:34.832252 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-24cfea9d-5dff57b685-lvzz4"] Sep 11 12:07:34.842489 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:34.842461 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-24cfea9d-5dff57b685-lvzz4"] Sep 11 12:07:36.337176 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:36.337139 2566 patch_prober.go:28] interesting pod/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": read tcp 10.132.0.2:60356->10.132.0.42:15021: read: connection reset by peer" start-of-body= Sep 11 12:07:36.337563 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:36.337211 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" podUID="47192df9-4559-4286-a94e-1f01440c4c65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": read tcp 10.132.0.2:60356->10.132.0.42:15021: read: connection reset by peer" Sep 11 12:07:36.758688 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:36.758606 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="340dddb2-e23e-4a04-b274-efcd007be4b2" path="/var/lib/kubelet/pods/340dddb2-e23e-4a04-b274-efcd007be4b2/volumes" Sep 11 12:07:36.813701 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:36.813665 2566 generic.go:356] "Generic (PLEG): container finished" podID="47192df9-4559-4286-a94e-1f01440c4c65" containerID="f2ad890a0a45f606d2171a404cd28259f497956d3dab85c93803c892682aaa90" exitCode=0 Sep 11 12:07:36.813862 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:36.813743 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" event={"ID":"47192df9-4559-4286-a94e-1f01440c4c65","Type":"ContainerDied","Data":"f2ad890a0a45f606d2171a404cd28259f497956d3dab85c93803c892682aaa90"} Sep 11 12:07:36.814156 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:36.814142 2566 scope.go:117] "RemoveContainer" containerID="f2ad890a0a45f606d2171a404cd28259f497956d3dab85c93803c892682aaa90" Sep 11 12:07:36.815170 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:36.815142 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:07:36.815225 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:36.815212 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:07:36.815261 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:36.815249 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:07:37.823853 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:37.823813 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" event={"ID":"47192df9-4559-4286-a94e-1f01440c4c65","Type":"ContainerStarted","Data":"5f4f5fa08fcea6e38307b6572e87f6600899c99aac57f6cc6eb6b870717b05e9"} Sep 11 12:07:37.919392 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:37.919342 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" Sep 11 12:07:37.919847 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:37.919816 2566 patch_prober.go:28] interesting pod/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Sep 11 12:07:37.920005 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:37.919899 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" podUID="47192df9-4559-4286-a94e-1f01440c4c65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Sep 11 12:07:38.920282 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:38.920231 2566 patch_prober.go:28] interesting pod/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Sep 11 12:07:38.920807 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:38.920325 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" podUID="47192df9-4559-4286-a94e-1f01440c4c65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Sep 11 12:07:39.920167 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:39.920119 2566 patch_prober.go:28] interesting pod/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Sep 11 12:07:39.920390 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:39.920194 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" podUID="47192df9-4559-4286-a94e-1f01440c4c65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Sep 11 12:07:40.919691 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:40.919655 2566 patch_prober.go:28] interesting pod/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Sep 11 12:07:40.920030 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:40.919990 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" podUID="47192df9-4559-4286-a94e-1f01440c4c65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Sep 11 12:07:41.920169 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:41.920115 2566 patch_prober.go:28] interesting pod/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Sep 11 12:07:41.920685 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:41.920197 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" podUID="47192df9-4559-4286-a94e-1f01440c4c65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Sep 11 12:07:42.919383 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:42.919333 2566 patch_prober.go:28] interesting pod/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Sep 11 12:07:42.919607 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:42.919411 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" podUID="47192df9-4559-4286-a94e-1f01440c4c65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Sep 11 12:07:43.922685 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:43.922644 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" Sep 11 12:07:43.923185 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:43.923169 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" Sep 11 12:07:43.923808 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:43.923791 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" Sep 11 12:07:51.508507 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:51.508466 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d844cc-cc8b44746-lzlgq"] Sep 11 12:07:51.509011 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:51.508986 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="340dddb2-e23e-4a04-b274-efcd007be4b2" containerName="payload-processing" Sep 11 12:07:51.509011 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:51.509007 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="340dddb2-e23e-4a04-b274-efcd007be4b2" containerName="payload-processing" Sep 11 12:07:51.509216 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:51.509095 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="340dddb2-e23e-4a04-b274-efcd007be4b2" containerName="payload-processing" Sep 11 12:07:51.511972 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:51.511953 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d844cc-cc8b44746-lzlgq" Sep 11 12:07:51.514596 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:51.514576 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-d844cc\"" Sep 11 12:07:51.514932 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:51.514916 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-d844cc-dockercfg-n24j9\"" Sep 11 12:07:51.521214 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:51.521187 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d844cc-cc8b44746-lzlgq"] Sep 11 12:07:51.620253 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:51.620209 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc86f6e6-bbf2-43d6-ae13-16b3a1484b92-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-d844cc-cc8b44746-lzlgq\" (UID: \"cc86f6e6-bbf2-43d6-ae13-16b3a1484b92\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d844cc-cc8b44746-lzlgq" Sep 11 12:07:51.620441 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:51.620272 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whrpf\" (UniqueName: \"kubernetes.io/projected/cc86f6e6-bbf2-43d6-ae13-16b3a1484b92-kube-api-access-whrpf\") pod \"payload-pre-processing-e2e-shared-b-w3-d844cc-cc8b44746-lzlgq\" (UID: \"cc86f6e6-bbf2-43d6-ae13-16b3a1484b92\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d844cc-cc8b44746-lzlgq" Sep 11 12:07:51.721005 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:51.720965 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc86f6e6-bbf2-43d6-ae13-16b3a1484b92-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-d844cc-cc8b44746-lzlgq\" (UID: \"cc86f6e6-bbf2-43d6-ae13-16b3a1484b92\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d844cc-cc8b44746-lzlgq" Sep 11 12:07:51.721005 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:51.721011 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whrpf\" (UniqueName: \"kubernetes.io/projected/cc86f6e6-bbf2-43d6-ae13-16b3a1484b92-kube-api-access-whrpf\") pod \"payload-pre-processing-e2e-shared-b-w3-d844cc-cc8b44746-lzlgq\" (UID: \"cc86f6e6-bbf2-43d6-ae13-16b3a1484b92\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d844cc-cc8b44746-lzlgq" Sep 11 12:07:51.721786 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:51.721763 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc86f6e6-bbf2-43d6-ae13-16b3a1484b92-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-d844cc-cc8b44746-lzlgq\" (UID: \"cc86f6e6-bbf2-43d6-ae13-16b3a1484b92\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d844cc-cc8b44746-lzlgq" Sep 11 12:07:51.730629 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:51.730592 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whrpf\" (UniqueName: \"kubernetes.io/projected/cc86f6e6-bbf2-43d6-ae13-16b3a1484b92-kube-api-access-whrpf\") pod \"payload-pre-processing-e2e-shared-b-w3-d844cc-cc8b44746-lzlgq\" (UID: \"cc86f6e6-bbf2-43d6-ae13-16b3a1484b92\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d844cc-cc8b44746-lzlgq" Sep 11 12:07:51.822814 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:51.822776 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d844cc-cc8b44746-lzlgq" Sep 11 12:07:52.113106 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:52.113077 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d844cc-cc8b44746-lzlgq"] Sep 11 12:07:52.114747 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:07:52.114718 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc86f6e6_bbf2_43d6_ae13_16b3a1484b92.slice/crio-e9c80081afeb3bb572f1b5bfcf6936d0e5beda40d20d7beebb7fa89ece39a41a WatchSource:0}: Error finding container e9c80081afeb3bb572f1b5bfcf6936d0e5beda40d20d7beebb7fa89ece39a41a: Status 404 returned error can't find the container with id e9c80081afeb3bb572f1b5bfcf6936d0e5beda40d20d7beebb7fa89ece39a41a Sep 11 12:07:52.904600 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:52.904564 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d844cc-cc8b44746-lzlgq" event={"ID":"cc86f6e6-bbf2-43d6-ae13-16b3a1484b92","Type":"ContainerStarted","Data":"13df99bc8a44cc3aa94fcc19f381566c71956074aad684604db99012887ff9a9"} Sep 11 12:07:52.905045 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:52.904619 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d844cc-cc8b44746-lzlgq" event={"ID":"cc86f6e6-bbf2-43d6-ae13-16b3a1484b92","Type":"ContainerStarted","Data":"e9c80081afeb3bb572f1b5bfcf6936d0e5beda40d20d7beebb7fa89ece39a41a"} Sep 11 12:07:52.905045 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:52.904698 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d844cc-cc8b44746-lzlgq" Sep 11 12:07:52.934353 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:07:52.934291 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d844cc-cc8b44746-lzlgq" podStartSLOduration=1.9342703719999998 podStartE2EDuration="1.934270372s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:07:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:07:52.934164154 +0000 UTC m=+1204.699137328" watchObservedRunningTime="2026-09-11 12:07:52.934270372 +0000 UTC m=+1204.699243547" Sep 11 12:08:02.337468 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:02.337428 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-45352b-795454fb55-wh6kd"] Sep 11 12:08:02.339940 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:02.339917 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-45352b-795454fb55-wh6kd" Sep 11 12:08:02.343018 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:02.342993 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-45352b\"" Sep 11 12:08:02.343255 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:02.343233 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-45352b-dockercfg-xxx7x\"" Sep 11 12:08:02.356558 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:02.356531 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-45352b-795454fb55-wh6kd"] Sep 11 12:08:02.421429 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:02.421392 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7l6f\" (UniqueName: \"kubernetes.io/projected/503e63a3-d0bb-4852-8032-ef70c35dd673-kube-api-access-r7l6f\") pod \"payload-processing-e2e-shared-a-w2-45352b-795454fb55-wh6kd\" (UID: \"503e63a3-d0bb-4852-8032-ef70c35dd673\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-45352b-795454fb55-wh6kd" Sep 11 12:08:02.421660 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:02.421449 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/503e63a3-d0bb-4852-8032-ef70c35dd673-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-45352b-795454fb55-wh6kd\" (UID: \"503e63a3-d0bb-4852-8032-ef70c35dd673\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-45352b-795454fb55-wh6kd" Sep 11 12:08:02.522402 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:02.522361 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7l6f\" (UniqueName: \"kubernetes.io/projected/503e63a3-d0bb-4852-8032-ef70c35dd673-kube-api-access-r7l6f\") pod \"payload-processing-e2e-shared-a-w2-45352b-795454fb55-wh6kd\" (UID: \"503e63a3-d0bb-4852-8032-ef70c35dd673\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-45352b-795454fb55-wh6kd" Sep 11 12:08:02.522584 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:02.522422 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/503e63a3-d0bb-4852-8032-ef70c35dd673-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-45352b-795454fb55-wh6kd\" (UID: \"503e63a3-d0bb-4852-8032-ef70c35dd673\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-45352b-795454fb55-wh6kd" Sep 11 12:08:02.523197 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:02.523177 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/503e63a3-d0bb-4852-8032-ef70c35dd673-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-45352b-795454fb55-wh6kd\" (UID: \"503e63a3-d0bb-4852-8032-ef70c35dd673\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-45352b-795454fb55-wh6kd" Sep 11 12:08:02.532777 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:02.532743 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7l6f\" (UniqueName: \"kubernetes.io/projected/503e63a3-d0bb-4852-8032-ef70c35dd673-kube-api-access-r7l6f\") pod \"payload-processing-e2e-shared-a-w2-45352b-795454fb55-wh6kd\" (UID: \"503e63a3-d0bb-4852-8032-ef70c35dd673\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-45352b-795454fb55-wh6kd" Sep 11 12:08:02.650995 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:02.650903 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-45352b-795454fb55-wh6kd" Sep 11 12:08:02.924068 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:02.924042 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-45352b-795454fb55-wh6kd"] Sep 11 12:08:02.926156 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:08:02.926120 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod503e63a3_d0bb_4852_8032_ef70c35dd673.slice/crio-b4e79451d5e95b18bbd993c1e1d007451038714e64a244e2786f4044fef997fc WatchSource:0}: Error finding container b4e79451d5e95b18bbd993c1e1d007451038714e64a244e2786f4044fef997fc: Status 404 returned error can't find the container with id b4e79451d5e95b18bbd993c1e1d007451038714e64a244e2786f4044fef997fc Sep 11 12:08:02.952033 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:02.951999 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-45352b-795454fb55-wh6kd" event={"ID":"503e63a3-d0bb-4852-8032-ef70c35dd673","Type":"ContainerStarted","Data":"b4e79451d5e95b18bbd993c1e1d007451038714e64a244e2786f4044fef997fc"} Sep 11 12:08:03.912782 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:03.912739 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d844cc-cc8b44746-lzlgq" Sep 11 12:08:03.959769 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:03.959730 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-45352b-795454fb55-wh6kd" event={"ID":"503e63a3-d0bb-4852-8032-ef70c35dd673","Type":"ContainerStarted","Data":"2926bb220d7a932bc39af6b9d01245a68985f523152cd2e24b50d335d3b5bec1"} Sep 11 12:08:03.959990 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:03.959899 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-45352b-795454fb55-wh6kd" Sep 11 12:08:03.980850 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:03.980777 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-45352b-795454fb55-wh6kd" podStartSLOduration=1.980742738 podStartE2EDuration="1.980742738s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:08:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:08:03.980185058 +0000 UTC m=+1215.745158234" watchObservedRunningTime="2026-09-11 12:08:03.980742738 +0000 UTC m=+1215.745715916" Sep 11 12:08:14.966743 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:14.966711 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-45352b-795454fb55-wh6kd" Sep 11 12:08:26.031157 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:26.031124 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58"] Sep 11 12:08:26.033589 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:26.033557 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" Sep 11 12:08:26.036544 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:26.036525 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-f7851b-openshift-default-dockercfg-wqh5z\"" Sep 11 12:08:26.049548 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:26.049522 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58"] Sep 11 12:08:26.129681 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:26.129638 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7a938ed-a372-4280-84a8-33877f1fd3be-istio-token\") pod \"e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58\" (UID: \"f7a938ed-a372-4280-84a8-33877f1fd3be\") " pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" Sep 11 12:08:26.129927 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:26.129699 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtgfv\" (UniqueName: \"kubernetes.io/projected/f7a938ed-a372-4280-84a8-33877f1fd3be-kube-api-access-mtgfv\") pod \"e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58\" (UID: \"f7a938ed-a372-4280-84a8-33877f1fd3be\") " pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" Sep 11 12:08:26.129927 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:26.129801 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7a938ed-a372-4280-84a8-33877f1fd3be-istiod-ca-cert\") pod \"e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58\" (UID: \"f7a938ed-a372-4280-84a8-33877f1fd3be\") " pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" Sep 11 12:08:26.129927 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:26.129887 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7a938ed-a372-4280-84a8-33877f1fd3be-istio-envoy\") pod \"e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58\" (UID: \"f7a938ed-a372-4280-84a8-33877f1fd3be\") " pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" Sep 11 12:08:26.129927 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:26.129924 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7a938ed-a372-4280-84a8-33877f1fd3be-istio-podinfo\") pod \"e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58\" (UID: \"f7a938ed-a372-4280-84a8-33877f1fd3be\") " pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" Sep 11 12:08:26.130224 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:26.130005 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7a938ed-a372-4280-84a8-33877f1fd3be-workload-certs\") pod \"e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58\" (UID: \"f7a938ed-a372-4280-84a8-33877f1fd3be\") " pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" Sep 11 12:08:26.130224 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:26.130058 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7a938ed-a372-4280-84a8-33877f1fd3be-istio-data\") pod \"e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58\" (UID: \"f7a938ed-a372-4280-84a8-33877f1fd3be\") " pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" Sep 11 12:08:26.130224 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:26.130099 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7a938ed-a372-4280-84a8-33877f1fd3be-credential-socket\") pod \"e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58\" (UID: \"f7a938ed-a372-4280-84a8-33877f1fd3be\") " pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" Sep 11 12:08:26.130224 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:26.130141 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7a938ed-a372-4280-84a8-33877f1fd3be-workload-socket\") pod \"e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58\" (UID: \"f7a938ed-a372-4280-84a8-33877f1fd3be\") " pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" Sep 11 12:08:26.231355 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:26.231315 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7a938ed-a372-4280-84a8-33877f1fd3be-istiod-ca-cert\") pod \"e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58\" (UID: \"f7a938ed-a372-4280-84a8-33877f1fd3be\") " pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" Sep 11 12:08:26.231355 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:26.231362 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7a938ed-a372-4280-84a8-33877f1fd3be-istio-envoy\") pod \"e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58\" (UID: \"f7a938ed-a372-4280-84a8-33877f1fd3be\") " pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" Sep 11 12:08:26.231629 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:26.231383 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7a938ed-a372-4280-84a8-33877f1fd3be-istio-podinfo\") pod \"e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58\" (UID: \"f7a938ed-a372-4280-84a8-33877f1fd3be\") " pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" Sep 11 12:08:26.231629 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:26.231418 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7a938ed-a372-4280-84a8-33877f1fd3be-workload-certs\") pod \"e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58\" (UID: \"f7a938ed-a372-4280-84a8-33877f1fd3be\") " pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" Sep 11 12:08:26.231629 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:26.231450 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7a938ed-a372-4280-84a8-33877f1fd3be-istio-data\") pod \"e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58\" (UID: \"f7a938ed-a372-4280-84a8-33877f1fd3be\") " pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" Sep 11 12:08:26.231629 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:26.231472 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7a938ed-a372-4280-84a8-33877f1fd3be-credential-socket\") pod \"e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58\" (UID: \"f7a938ed-a372-4280-84a8-33877f1fd3be\") " pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" Sep 11 12:08:26.231629 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:26.231504 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7a938ed-a372-4280-84a8-33877f1fd3be-workload-socket\") pod \"e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58\" (UID: \"f7a938ed-a372-4280-84a8-33877f1fd3be\") " pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" Sep 11 12:08:26.231629 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:26.231547 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7a938ed-a372-4280-84a8-33877f1fd3be-istio-token\") pod \"e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58\" (UID: \"f7a938ed-a372-4280-84a8-33877f1fd3be\") " pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" Sep 11 12:08:26.232067 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:26.231652 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtgfv\" (UniqueName: \"kubernetes.io/projected/f7a938ed-a372-4280-84a8-33877f1fd3be-kube-api-access-mtgfv\") pod \"e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58\" (UID: \"f7a938ed-a372-4280-84a8-33877f1fd3be\") " pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" Sep 11 12:08:26.232067 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:26.231937 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7a938ed-a372-4280-84a8-33877f1fd3be-credential-socket\") pod \"e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58\" (UID: \"f7a938ed-a372-4280-84a8-33877f1fd3be\") " pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" Sep 11 12:08:26.232067 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:26.231985 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7a938ed-a372-4280-84a8-33877f1fd3be-workload-socket\") pod \"e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58\" (UID: \"f7a938ed-a372-4280-84a8-33877f1fd3be\") " pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" Sep 11 12:08:26.232067 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:26.232002 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7a938ed-a372-4280-84a8-33877f1fd3be-workload-certs\") pod \"e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58\" (UID: \"f7a938ed-a372-4280-84a8-33877f1fd3be\") " pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" Sep 11 12:08:26.232374 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:26.232294 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7a938ed-a372-4280-84a8-33877f1fd3be-istio-data\") pod \"e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58\" (UID: \"f7a938ed-a372-4280-84a8-33877f1fd3be\") " pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" Sep 11 12:08:26.232374 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:26.232299 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7a938ed-a372-4280-84a8-33877f1fd3be-istiod-ca-cert\") pod \"e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58\" (UID: \"f7a938ed-a372-4280-84a8-33877f1fd3be\") " pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" Sep 11 12:08:26.233955 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:26.233932 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7a938ed-a372-4280-84a8-33877f1fd3be-istio-envoy\") pod \"e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58\" (UID: \"f7a938ed-a372-4280-84a8-33877f1fd3be\") " pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" Sep 11 12:08:26.234207 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:26.234147 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7a938ed-a372-4280-84a8-33877f1fd3be-istio-podinfo\") pod \"e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58\" (UID: \"f7a938ed-a372-4280-84a8-33877f1fd3be\") " pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" Sep 11 12:08:26.242084 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:26.242064 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7a938ed-a372-4280-84a8-33877f1fd3be-istio-token\") pod \"e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58\" (UID: \"f7a938ed-a372-4280-84a8-33877f1fd3be\") " pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" Sep 11 12:08:26.242445 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:26.242310 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtgfv\" (UniqueName: \"kubernetes.io/projected/f7a938ed-a372-4280-84a8-33877f1fd3be-kube-api-access-mtgfv\") pod \"e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58\" (UID: \"f7a938ed-a372-4280-84a8-33877f1fd3be\") " pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" Sep 11 12:08:26.344148 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:26.344114 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" Sep 11 12:08:26.621470 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:26.621304 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58"] Sep 11 12:08:26.624206 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:08:26.624178 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7a938ed_a372_4280_84a8_33877f1fd3be.slice/crio-8ccbc87b00ad977b805aaea8d88f2509d26fbe41259583ab7f56ae1c34dd4bc9 WatchSource:0}: Error finding container 8ccbc87b00ad977b805aaea8d88f2509d26fbe41259583ab7f56ae1c34dd4bc9: Status 404 returned error can't find the container with id 8ccbc87b00ad977b805aaea8d88f2509d26fbe41259583ab7f56ae1c34dd4bc9 Sep 11 12:08:26.626423 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:26.626389 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:08:26.626511 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:26.626457 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:08:26.626511 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:26.626486 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:08:27.072558 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:27.072515 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" event={"ID":"f7a938ed-a372-4280-84a8-33877f1fd3be","Type":"ContainerStarted","Data":"ebe06adafe28318f35f81150bffc1e25e6c9ce723a493c95946f04b51088d1be"} Sep 11 12:08:27.072977 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:27.072586 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" event={"ID":"f7a938ed-a372-4280-84a8-33877f1fd3be","Type":"ContainerStarted","Data":"8ccbc87b00ad977b805aaea8d88f2509d26fbe41259583ab7f56ae1c34dd4bc9"} Sep 11 12:08:27.098183 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:27.098127 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" podStartSLOduration=1.098109959 podStartE2EDuration="1.098109959s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:08:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:08:27.096607217 +0000 UTC m=+1238.861580393" watchObservedRunningTime="2026-09-11 12:08:27.098109959 +0000 UTC m=+1238.863083134" Sep 11 12:08:27.345367 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:27.345274 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" Sep 11 12:08:27.350587 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:27.350558 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" Sep 11 12:08:28.077629 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:28.077600 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" Sep 11 12:08:28.078597 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:28.078575 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" Sep 11 12:08:31.122404 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:31.122358 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7851b-69678bc8b6-rg8fq"] Sep 11 12:08:31.124737 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:31.124720 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7851b-69678bc8b6-rg8fq" Sep 11 12:08:31.127346 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:31.127322 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-f7851b-dockercfg-xxw4b\"" Sep 11 12:08:31.127652 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:31.127633 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-f7851b\"" Sep 11 12:08:31.139561 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:31.139536 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7851b-69678bc8b6-rg8fq"] Sep 11 12:08:31.181710 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:31.181676 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx4q7\" (UniqueName: \"kubernetes.io/projected/6b3221af-e568-460b-88fa-a0f1151c3772-kube-api-access-vx4q7\") pod \"payload-pre-processing-e2e-shared-b-w2-f7851b-69678bc8b6-rg8fq\" (UID: \"6b3221af-e568-460b-88fa-a0f1151c3772\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7851b-69678bc8b6-rg8fq" Sep 11 12:08:31.181860 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:31.181724 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b3221af-e568-460b-88fa-a0f1151c3772-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-f7851b-69678bc8b6-rg8fq\" (UID: \"6b3221af-e568-460b-88fa-a0f1151c3772\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7851b-69678bc8b6-rg8fq" Sep 11 12:08:31.282183 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:31.282141 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx4q7\" (UniqueName: \"kubernetes.io/projected/6b3221af-e568-460b-88fa-a0f1151c3772-kube-api-access-vx4q7\") pod \"payload-pre-processing-e2e-shared-b-w2-f7851b-69678bc8b6-rg8fq\" (UID: \"6b3221af-e568-460b-88fa-a0f1151c3772\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7851b-69678bc8b6-rg8fq" Sep 11 12:08:31.282383 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:31.282209 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b3221af-e568-460b-88fa-a0f1151c3772-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-f7851b-69678bc8b6-rg8fq\" (UID: \"6b3221af-e568-460b-88fa-a0f1151c3772\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7851b-69678bc8b6-rg8fq" Sep 11 12:08:31.283004 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:31.282977 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b3221af-e568-460b-88fa-a0f1151c3772-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-f7851b-69678bc8b6-rg8fq\" (UID: \"6b3221af-e568-460b-88fa-a0f1151c3772\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7851b-69678bc8b6-rg8fq" Sep 11 12:08:31.291683 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:31.291653 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx4q7\" (UniqueName: \"kubernetes.io/projected/6b3221af-e568-460b-88fa-a0f1151c3772-kube-api-access-vx4q7\") pod \"payload-pre-processing-e2e-shared-b-w2-f7851b-69678bc8b6-rg8fq\" (UID: \"6b3221af-e568-460b-88fa-a0f1151c3772\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7851b-69678bc8b6-rg8fq" Sep 11 12:08:31.435578 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:31.435481 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7851b-69678bc8b6-rg8fq" Sep 11 12:08:31.711263 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:31.711234 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7851b-69678bc8b6-rg8fq"] Sep 11 12:08:31.713644 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:08:31.713609 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b3221af_e568_460b_88fa_a0f1151c3772.slice/crio-aa78eb0cb6e68cdc3a7f136a7cfdea0dc1d41e30b618e1207f96d1b8ee49380a WatchSource:0}: Error finding container aa78eb0cb6e68cdc3a7f136a7cfdea0dc1d41e30b618e1207f96d1b8ee49380a: Status 404 returned error can't find the container with id aa78eb0cb6e68cdc3a7f136a7cfdea0dc1d41e30b618e1207f96d1b8ee49380a Sep 11 12:08:32.098951 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:32.098900 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7851b-69678bc8b6-rg8fq" event={"ID":"6b3221af-e568-460b-88fa-a0f1151c3772","Type":"ContainerStarted","Data":"c9180437c8dd8fc3b04f786bb272e65397934c3877890588639500abb2858655"} Sep 11 12:08:32.099164 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:32.098968 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7851b-69678bc8b6-rg8fq" event={"ID":"6b3221af-e568-460b-88fa-a0f1151c3772","Type":"ContainerStarted","Data":"aa78eb0cb6e68cdc3a7f136a7cfdea0dc1d41e30b618e1207f96d1b8ee49380a"} Sep 11 12:08:32.099164 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:32.099013 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7851b-69678bc8b6-rg8fq" Sep 11 12:08:32.118521 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:32.118457 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7851b-69678bc8b6-rg8fq" podStartSLOduration=1.118437065 podStartE2EDuration="1.118437065s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:08:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:08:32.116129874 +0000 UTC m=+1243.881103048" watchObservedRunningTime="2026-09-11 12:08:32.118437065 +0000 UTC m=+1243.883410244" Sep 11 12:08:43.107166 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:43.107073 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7851b-69678bc8b6-rg8fq" Sep 11 12:08:48.181220 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:48.181185 2566 generic.go:356] "Generic (PLEG): container finished" podID="f7a938ed-a372-4280-84a8-33877f1fd3be" containerID="ebe06adafe28318f35f81150bffc1e25e6c9ce723a493c95946f04b51088d1be" exitCode=0 Sep 11 12:08:48.181625 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:48.181255 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" event={"ID":"f7a938ed-a372-4280-84a8-33877f1fd3be","Type":"ContainerDied","Data":"ebe06adafe28318f35f81150bffc1e25e6c9ce723a493c95946f04b51088d1be"} Sep 11 12:08:48.181668 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:48.181647 2566 scope.go:117] "RemoveContainer" containerID="ebe06adafe28318f35f81150bffc1e25e6c9ce723a493c95946f04b51088d1be" Sep 11 12:08:48.182837 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:48.182805 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:08:48.182938 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:48.182912 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:08:48.182981 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:48.182948 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:08:49.188076 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:49.188039 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" event={"ID":"f7a938ed-a372-4280-84a8-33877f1fd3be","Type":"ContainerStarted","Data":"108b966e56cbe0a7fc527c58481d455d61eb206fcb948c0869c0f48b4eed9a05"} Sep 11 12:08:49.344581 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:49.344533 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" Sep 11 12:08:49.344822 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:49.344795 2566 patch_prober.go:28] interesting pod/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Sep 11 12:08:49.344972 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:49.344851 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" podUID="f7a938ed-a372-4280-84a8-33877f1fd3be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Sep 11 12:08:50.345221 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:50.345173 2566 patch_prober.go:28] interesting pod/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Sep 11 12:08:50.345688 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:50.345240 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" podUID="f7a938ed-a372-4280-84a8-33877f1fd3be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Sep 11 12:08:51.344490 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:51.344451 2566 patch_prober.go:28] interesting pod/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Sep 11 12:08:51.344666 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:51.344517 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" podUID="f7a938ed-a372-4280-84a8-33877f1fd3be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Sep 11 12:08:52.345200 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:52.345161 2566 patch_prober.go:28] interesting pod/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Sep 11 12:08:52.345578 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:52.345230 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" podUID="f7a938ed-a372-4280-84a8-33877f1fd3be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Sep 11 12:08:53.345275 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:53.345231 2566 patch_prober.go:28] interesting pod/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Sep 11 12:08:53.345670 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:53.345319 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" podUID="f7a938ed-a372-4280-84a8-33877f1fd3be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Sep 11 12:08:54.344464 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:54.344427 2566 patch_prober.go:28] interesting pod/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Sep 11 12:08:54.344853 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:54.344483 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" podUID="f7a938ed-a372-4280-84a8-33877f1fd3be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Sep 11 12:08:54.676159 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:54.676075 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7462-6a8a08"] Sep 11 12:08:54.678484 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:54.678465 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7462-6a8a08" Sep 11 12:08:54.680951 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:54.680927 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-wkgmv\"" Sep 11 12:08:54.687358 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:54.687331 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7462-6a8a08"] Sep 11 12:08:54.796342 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:54.796309 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w264q\" (UniqueName: \"kubernetes.io/projected/c998e186-3248-43fc-a9a8-d4ff4c0113b9-kube-api-access-w264q\") pod \"test-curl-7462-6a8a08\" (UID: \"c998e186-3248-43fc-a9a8-d4ff4c0113b9\") " pod="opendatahub/test-curl-7462-6a8a08" Sep 11 12:08:54.897258 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:54.897205 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w264q\" (UniqueName: \"kubernetes.io/projected/c998e186-3248-43fc-a9a8-d4ff4c0113b9-kube-api-access-w264q\") pod \"test-curl-7462-6a8a08\" (UID: \"c998e186-3248-43fc-a9a8-d4ff4c0113b9\") " pod="opendatahub/test-curl-7462-6a8a08" Sep 11 12:08:54.905798 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:54.905766 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w264q\" (UniqueName: \"kubernetes.io/projected/c998e186-3248-43fc-a9a8-d4ff4c0113b9-kube-api-access-w264q\") pod \"test-curl-7462-6a8a08\" (UID: \"c998e186-3248-43fc-a9a8-d4ff4c0113b9\") " pod="opendatahub/test-curl-7462-6a8a08" Sep 11 12:08:54.988434 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:54.988337 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7462-6a8a08" Sep 11 12:08:55.260508 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:55.260454 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7462-6a8a08"] Sep 11 12:08:55.262857 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:08:55.262831 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc998e186_3248_43fc_a9a8_d4ff4c0113b9.slice/crio-f7aea3ddad425cbd5c65c595a95e41a799224506e254c5f30c27e3ff31f7bbd1 WatchSource:0}: Error finding container f7aea3ddad425cbd5c65c595a95e41a799224506e254c5f30c27e3ff31f7bbd1: Status 404 returned error can't find the container with id f7aea3ddad425cbd5c65c595a95e41a799224506e254c5f30c27e3ff31f7bbd1 Sep 11 12:08:55.265094 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:55.265075 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 11 12:08:55.348412 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:55.348379 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" Sep 11 12:08:55.348683 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:55.348665 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" Sep 11 12:08:55.349307 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:55.349289 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" Sep 11 12:08:56.228591 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:56.228552 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7462-6a8a08" event={"ID":"c998e186-3248-43fc-a9a8-d4ff4c0113b9","Type":"ContainerStarted","Data":"f7aea3ddad425cbd5c65c595a95e41a799224506e254c5f30c27e3ff31f7bbd1"} Sep 11 12:08:57.234822 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:57.234777 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7462-6a8a08" event={"ID":"c998e186-3248-43fc-a9a8-d4ff4c0113b9","Type":"ContainerStarted","Data":"fc697c6194fc238cf3f58db6b717992da44937060dbf07688ef22f6cdb03e5cf"} Sep 11 12:08:57.251489 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:57.251439 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7462-6a8a08" podStartSLOduration=1.471777345 podStartE2EDuration="3.251424749s" totalImagesPullingTime="1.779647404s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:08:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:08:57.250460589 +0000 UTC m=+1269.015433765" watchObservedRunningTime="2026-09-11 12:08:57.251424749 +0000 UTC m=+1269.016397923" Sep 11 12:08:57.839268 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:57.839236 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7462-6a8a08"] Sep 11 12:08:57.843163 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:57.843132 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7462-6a8a08"] Sep 11 12:08:57.978941 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:57.978904 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7462-e81687"] Sep 11 12:08:57.979321 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:57.979302 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c998e186-3248-43fc-a9a8-d4ff4c0113b9" containerName="test-curl-7462-6a8a08" Sep 11 12:08:57.979405 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:57.979324 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c998e186-3248-43fc-a9a8-d4ff4c0113b9" containerName="test-curl-7462-6a8a08" Sep 11 12:08:57.979459 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:57.979431 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c998e186-3248-43fc-a9a8-d4ff4c0113b9" containerName="test-curl-7462-6a8a08" Sep 11 12:08:57.981327 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:57.981307 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7462-e81687" Sep 11 12:08:58.002859 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:58.002829 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7462-e81687"] Sep 11 12:08:58.132259 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:58.132164 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mptjm\" (UniqueName: \"kubernetes.io/projected/3d30c943-d018-4150-bc78-92cb92fbb4ac-kube-api-access-mptjm\") pod \"test-curl-7462-e81687\" (UID: \"3d30c943-d018-4150-bc78-92cb92fbb4ac\") " pod="opendatahub/test-curl-7462-e81687" Sep 11 12:08:58.233215 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:58.233175 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mptjm\" (UniqueName: \"kubernetes.io/projected/3d30c943-d018-4150-bc78-92cb92fbb4ac-kube-api-access-mptjm\") pod \"test-curl-7462-e81687\" (UID: \"3d30c943-d018-4150-bc78-92cb92fbb4ac\") " pod="opendatahub/test-curl-7462-e81687" Sep 11 12:08:58.240058 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:08:58.240029 2566 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7462-6a8a08" container="test-curl-7462-6a8a08" Sep 11 12:08:58.242796 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:58.242771 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mptjm\" (UniqueName: \"kubernetes.io/projected/3d30c943-d018-4150-bc78-92cb92fbb4ac-kube-api-access-mptjm\") pod \"test-curl-7462-e81687\" (UID: \"3d30c943-d018-4150-bc78-92cb92fbb4ac\") " pod="opendatahub/test-curl-7462-e81687" Sep 11 12:08:58.289759 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:58.289726 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7462-e81687" Sep 11 12:08:58.565432 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:58.565406 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7462-e81687"] Sep 11 12:08:58.567511 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:08:58.567487 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d30c943_d018_4150_bc78_92cb92fbb4ac.slice/crio-fc77a828d1e77aa7e2b73c35ff3739edd642c9d262c99c386b6b50c460c1c114 WatchSource:0}: Error finding container fc77a828d1e77aa7e2b73c35ff3739edd642c9d262c99c386b6b50c460c1c114: Status 404 returned error can't find the container with id fc77a828d1e77aa7e2b73c35ff3739edd642c9d262c99c386b6b50c460c1c114 Sep 11 12:08:59.245287 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:59.245248 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7462-e81687" event={"ID":"3d30c943-d018-4150-bc78-92cb92fbb4ac","Type":"ContainerStarted","Data":"ce605004132a094b31ceb8717b79f1d0e481bd3b8818e2ad62b92e2ad8bada33"} Sep 11 12:08:59.245744 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:59.245297 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7462-e81687" event={"ID":"3d30c943-d018-4150-bc78-92cb92fbb4ac","Type":"ContainerStarted","Data":"fc77a828d1e77aa7e2b73c35ff3739edd642c9d262c99c386b6b50c460c1c114"} Sep 11 12:08:59.245744 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:59.245398 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7462-6a8a08" podUID="c998e186-3248-43fc-a9a8-d4ff4c0113b9" containerName="test-curl-7462-6a8a08" containerID="cri-o://fc697c6194fc238cf3f58db6b717992da44937060dbf07688ef22f6cdb03e5cf" gracePeriod=2 Sep 11 12:08:59.261292 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:59.261240 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7462-e81687" podStartSLOduration=2.261224574 podStartE2EDuration="2.261224574s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:08:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:08:59.260834951 +0000 UTC m=+1271.025808128" watchObservedRunningTime="2026-09-11 12:08:59.261224574 +0000 UTC m=+1271.026197797" Sep 11 12:08:59.906029 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:59.905993 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7462-e81687"] Sep 11 12:08:59.909748 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:08:59.909720 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7462-e81687"] Sep 11 12:09:00.025426 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:00.025335 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7462-e91c31"] Sep 11 12:09:00.025738 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:00.025723 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d30c943-d018-4150-bc78-92cb92fbb4ac" containerName="test-curl-7462-e81687" Sep 11 12:09:00.025785 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:00.025741 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d30c943-d018-4150-bc78-92cb92fbb4ac" containerName="test-curl-7462-e81687" Sep 11 12:09:00.025821 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:00.025807 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d30c943-d018-4150-bc78-92cb92fbb4ac" containerName="test-curl-7462-e81687" Sep 11 12:09:00.027969 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:00.027948 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7462-e91c31" Sep 11 12:09:00.034348 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:00.034318 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7462-e91c31"] Sep 11 12:09:00.151035 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:00.150994 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwb2p\" (UniqueName: \"kubernetes.io/projected/f68dbcdf-e52f-40e0-b5c6-35e7cac20379-kube-api-access-gwb2p\") pod \"test-curl-7462-e91c31\" (UID: \"f68dbcdf-e52f-40e0-b5c6-35e7cac20379\") " pod="opendatahub/test-curl-7462-e91c31" Sep 11 12:09:00.250313 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:09:00.250279 2566 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7462-e81687" container="test-curl-7462-e81687" Sep 11 12:09:00.251599 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:00.251574 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwb2p\" (UniqueName: \"kubernetes.io/projected/f68dbcdf-e52f-40e0-b5c6-35e7cac20379-kube-api-access-gwb2p\") pod \"test-curl-7462-e91c31\" (UID: \"f68dbcdf-e52f-40e0-b5c6-35e7cac20379\") " pod="opendatahub/test-curl-7462-e91c31" Sep 11 12:09:00.261251 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:00.261222 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwb2p\" (UniqueName: \"kubernetes.io/projected/f68dbcdf-e52f-40e0-b5c6-35e7cac20379-kube-api-access-gwb2p\") pod \"test-curl-7462-e91c31\" (UID: \"f68dbcdf-e52f-40e0-b5c6-35e7cac20379\") " pod="opendatahub/test-curl-7462-e91c31" Sep 11 12:09:00.338248 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:00.338195 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7462-e91c31" Sep 11 12:09:00.606414 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:00.606337 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7462-e91c31"] Sep 11 12:09:00.609717 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:09:00.609677 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf68dbcdf_e52f_40e0_b5c6_35e7cac20379.slice/crio-c1354fd9413a26f92ee90dc96f1968115e9b0ba9d962bdf26b051912a8cc4480 WatchSource:0}: Error finding container c1354fd9413a26f92ee90dc96f1968115e9b0ba9d962bdf26b051912a8cc4480: Status 404 returned error can't find the container with id c1354fd9413a26f92ee90dc96f1968115e9b0ba9d962bdf26b051912a8cc4480 Sep 11 12:09:01.256332 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:01.256309 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7462-e91c31" event={"ID":"f68dbcdf-e52f-40e0-b5c6-35e7cac20379","Type":"ContainerStarted","Data":"5e3fe7a2aa0352d8244a1b848325b221c165af8086f915bb3022bf325402b62d"} Sep 11 12:09:01.256624 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:01.256347 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7462-e91c31" event={"ID":"f68dbcdf-e52f-40e0-b5c6-35e7cac20379","Type":"ContainerStarted","Data":"c1354fd9413a26f92ee90dc96f1968115e9b0ba9d962bdf26b051912a8cc4480"} Sep 11 12:09:01.256624 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:01.256407 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7462-e81687" podUID="3d30c943-d018-4150-bc78-92cb92fbb4ac" containerName="test-curl-7462-e81687" containerID="cri-o://ce605004132a094b31ceb8717b79f1d0e481bd3b8818e2ad62b92e2ad8bada33" gracePeriod=2 Sep 11 12:09:01.273024 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:01.272983 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7462-e91c31" podStartSLOduration=1.272966141 podStartE2EDuration="1.272966141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:09:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:09:01.272517825 +0000 UTC m=+1273.037491001" watchObservedRunningTime="2026-09-11 12:09:01.272966141 +0000 UTC m=+1273.037939317" Sep 11 12:09:01.425626 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:01.425598 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7462-6a8a08" Sep 11 12:09:01.562351 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:01.562258 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w264q\" (UniqueName: \"kubernetes.io/projected/c998e186-3248-43fc-a9a8-d4ff4c0113b9-kube-api-access-w264q\") pod \"c998e186-3248-43fc-a9a8-d4ff4c0113b9\" (UID: \"c998e186-3248-43fc-a9a8-d4ff4c0113b9\") " Sep 11 12:09:01.564289 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:01.564258 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c998e186-3248-43fc-a9a8-d4ff4c0113b9-kube-api-access-w264q" (OuterVolumeSpecName: "kube-api-access-w264q") pod "c998e186-3248-43fc-a9a8-d4ff4c0113b9" (UID: "c998e186-3248-43fc-a9a8-d4ff4c0113b9"). InnerVolumeSpecName "kube-api-access-w264q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:09:01.663422 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:01.663396 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w264q\" (UniqueName: \"kubernetes.io/projected/c998e186-3248-43fc-a9a8-d4ff4c0113b9-kube-api-access-w264q\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:09:01.871746 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:01.871709 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7462-e91c31"] Sep 11 12:09:01.875752 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:01.875721 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7462-e91c31"] Sep 11 12:09:01.990096 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:01.990063 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7462-ef7341"] Sep 11 12:09:01.990419 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:01.990406 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f68dbcdf-e52f-40e0-b5c6-35e7cac20379" containerName="test-curl-7462-e91c31" Sep 11 12:09:01.990487 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:01.990421 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f68dbcdf-e52f-40e0-b5c6-35e7cac20379" containerName="test-curl-7462-e91c31" Sep 11 12:09:01.990487 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:01.990474 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f68dbcdf-e52f-40e0-b5c6-35e7cac20379" containerName="test-curl-7462-e91c31" Sep 11 12:09:01.992382 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:01.992366 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7462-ef7341" Sep 11 12:09:02.003099 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:02.003067 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7462-ef7341"] Sep 11 12:09:02.167618 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:02.167523 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-652f8\" (UniqueName: \"kubernetes.io/projected/0a45ba51-03da-4f7b-9acf-09f1a8035fed-kube-api-access-652f8\") pod \"test-curl-7462-ef7341\" (UID: \"0a45ba51-03da-4f7b-9acf-09f1a8035fed\") " pod="opendatahub/test-curl-7462-ef7341" Sep 11 12:09:02.262320 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:02.262284 2566 generic.go:356] "Generic (PLEG): container finished" podID="c998e186-3248-43fc-a9a8-d4ff4c0113b9" containerID="fc697c6194fc238cf3f58db6b717992da44937060dbf07688ef22f6cdb03e5cf" exitCode=137 Sep 11 12:09:02.262773 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:02.262333 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7462-6a8a08" Sep 11 12:09:02.262773 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:02.262349 2566 scope.go:117] "RemoveContainer" containerID="fc697c6194fc238cf3f58db6b717992da44937060dbf07688ef22f6cdb03e5cf" Sep 11 12:09:02.262773 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:09:02.262719 2566 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7462-e91c31" container="test-curl-7462-e91c31" Sep 11 12:09:02.268135 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:02.268111 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-652f8\" (UniqueName: \"kubernetes.io/projected/0a45ba51-03da-4f7b-9acf-09f1a8035fed-kube-api-access-652f8\") pod \"test-curl-7462-ef7341\" (UID: \"0a45ba51-03da-4f7b-9acf-09f1a8035fed\") " pod="opendatahub/test-curl-7462-ef7341" Sep 11 12:09:02.271049 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:02.271032 2566 scope.go:117] "RemoveContainer" containerID="fc697c6194fc238cf3f58db6b717992da44937060dbf07688ef22f6cdb03e5cf" Sep 11 12:09:02.271336 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:09:02.271317 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc697c6194fc238cf3f58db6b717992da44937060dbf07688ef22f6cdb03e5cf\": container with ID starting with fc697c6194fc238cf3f58db6b717992da44937060dbf07688ef22f6cdb03e5cf not found: ID does not exist" containerID="fc697c6194fc238cf3f58db6b717992da44937060dbf07688ef22f6cdb03e5cf" Sep 11 12:09:02.271384 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:02.271347 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc697c6194fc238cf3f58db6b717992da44937060dbf07688ef22f6cdb03e5cf"} err="failed to get container status \"fc697c6194fc238cf3f58db6b717992da44937060dbf07688ef22f6cdb03e5cf\": rpc error: code = NotFound desc = could not find container \"fc697c6194fc238cf3f58db6b717992da44937060dbf07688ef22f6cdb03e5cf\": container with ID starting with fc697c6194fc238cf3f58db6b717992da44937060dbf07688ef22f6cdb03e5cf not found: ID does not exist" Sep 11 12:09:02.272371 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:09:02.272346 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7462-6a8a08\" is forbidden: User \"system:node:ip-10-0-132-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-170.ec2.internal' and this object" podUID="c998e186-3248-43fc-a9a8-d4ff4c0113b9" pod="opendatahub/test-curl-7462-6a8a08" Sep 11 12:09:02.277661 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:02.277639 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-652f8\" (UniqueName: \"kubernetes.io/projected/0a45ba51-03da-4f7b-9acf-09f1a8035fed-kube-api-access-652f8\") pod \"test-curl-7462-ef7341\" (UID: \"0a45ba51-03da-4f7b-9acf-09f1a8035fed\") " pod="opendatahub/test-curl-7462-ef7341" Sep 11 12:09:02.301562 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:02.301531 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7462-ef7341" Sep 11 12:09:02.574740 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:02.574714 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7462-ef7341"] Sep 11 12:09:02.577180 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:09:02.577151 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0a45ba51_03da_4f7b_9acf_09f1a8035fed.slice/crio-4c04ca6dca53b479c072ae288b0f9d8eb198d9797fde104075e45bd0ffd768fd WatchSource:0}: Error finding container 4c04ca6dca53b479c072ae288b0f9d8eb198d9797fde104075e45bd0ffd768fd: Status 404 returned error can't find the container with id 4c04ca6dca53b479c072ae288b0f9d8eb198d9797fde104075e45bd0ffd768fd Sep 11 12:09:02.758646 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:02.758614 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c998e186-3248-43fc-a9a8-d4ff4c0113b9" path="/var/lib/kubelet/pods/c998e186-3248-43fc-a9a8-d4ff4c0113b9/volumes" Sep 11 12:09:03.268466 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:03.268432 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7462-ef7341" event={"ID":"0a45ba51-03da-4f7b-9acf-09f1a8035fed","Type":"ContainerStarted","Data":"977c3088f2b22d81d54d6d2a455da3c20b5eb7c7810f1c7f009c07419dc98650"} Sep 11 12:09:03.268821 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:03.268480 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7462-ef7341" event={"ID":"0a45ba51-03da-4f7b-9acf-09f1a8035fed","Type":"ContainerStarted","Data":"4c04ca6dca53b479c072ae288b0f9d8eb198d9797fde104075e45bd0ffd768fd"} Sep 11 12:09:03.269547 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:03.269522 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7462-e91c31" podUID="f68dbcdf-e52f-40e0-b5c6-35e7cac20379" containerName="test-curl-7462-e91c31" containerID="cri-o://5e3fe7a2aa0352d8244a1b848325b221c165af8086f915bb3022bf325402b62d" gracePeriod=2 Sep 11 12:09:03.285304 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:03.285260 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7462-ef7341" podStartSLOduration=2.285242748 podStartE2EDuration="2.285242748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:09:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:09:03.283265165 +0000 UTC m=+1275.048238342" watchObservedRunningTime="2026-09-11 12:09:03.285242748 +0000 UTC m=+1275.050215922" Sep 11 12:09:03.435214 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:03.435190 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7462-e81687" Sep 11 12:09:03.581152 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:03.581123 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mptjm\" (UniqueName: \"kubernetes.io/projected/3d30c943-d018-4150-bc78-92cb92fbb4ac-kube-api-access-mptjm\") pod \"3d30c943-d018-4150-bc78-92cb92fbb4ac\" (UID: \"3d30c943-d018-4150-bc78-92cb92fbb4ac\") " Sep 11 12:09:03.583189 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:03.583159 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d30c943-d018-4150-bc78-92cb92fbb4ac-kube-api-access-mptjm" (OuterVolumeSpecName: "kube-api-access-mptjm") pod "3d30c943-d018-4150-bc78-92cb92fbb4ac" (UID: "3d30c943-d018-4150-bc78-92cb92fbb4ac"). InnerVolumeSpecName "kube-api-access-mptjm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:09:03.681898 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:03.681835 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mptjm\" (UniqueName: \"kubernetes.io/projected/3d30c943-d018-4150-bc78-92cb92fbb4ac-kube-api-access-mptjm\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:09:03.857322 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:03.857233 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7462-ef7341"] Sep 11 12:09:03.865228 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:03.865195 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7462-ef7341"] Sep 11 12:09:04.275235 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:04.275139 2566 generic.go:356] "Generic (PLEG): container finished" podID="3d30c943-d018-4150-bc78-92cb92fbb4ac" containerID="ce605004132a094b31ceb8717b79f1d0e481bd3b8818e2ad62b92e2ad8bada33" exitCode=137 Sep 11 12:09:04.275235 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:04.275204 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7462-e81687" Sep 11 12:09:04.275744 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:04.275238 2566 scope.go:117] "RemoveContainer" containerID="ce605004132a094b31ceb8717b79f1d0e481bd3b8818e2ad62b92e2ad8bada33" Sep 11 12:09:04.275744 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:09:04.275553 2566 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7462-ef7341" container="test-curl-7462-ef7341" Sep 11 12:09:04.284184 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:04.284168 2566 scope.go:117] "RemoveContainer" containerID="ce605004132a094b31ceb8717b79f1d0e481bd3b8818e2ad62b92e2ad8bada33" Sep 11 12:09:04.284460 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:09:04.284439 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce605004132a094b31ceb8717b79f1d0e481bd3b8818e2ad62b92e2ad8bada33\": container with ID starting with ce605004132a094b31ceb8717b79f1d0e481bd3b8818e2ad62b92e2ad8bada33 not found: ID does not exist" containerID="ce605004132a094b31ceb8717b79f1d0e481bd3b8818e2ad62b92e2ad8bada33" Sep 11 12:09:04.284530 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:04.284472 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce605004132a094b31ceb8717b79f1d0e481bd3b8818e2ad62b92e2ad8bada33"} err="failed to get container status \"ce605004132a094b31ceb8717b79f1d0e481bd3b8818e2ad62b92e2ad8bada33\": rpc error: code = NotFound desc = could not find container \"ce605004132a094b31ceb8717b79f1d0e481bd3b8818e2ad62b92e2ad8bada33\": container with ID starting with ce605004132a094b31ceb8717b79f1d0e481bd3b8818e2ad62b92e2ad8bada33 not found: ID does not exist" Sep 11 12:09:04.285657 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:09:04.285634 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7462-e81687\" is forbidden: User \"system:node:ip-10-0-132-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-170.ec2.internal' and this object" podUID="3d30c943-d018-4150-bc78-92cb92fbb4ac" pod="opendatahub/test-curl-7462-e81687" Sep 11 12:09:04.720015 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:04.719983 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl"] Sep 11 12:09:04.720386 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:04.720369 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a45ba51-03da-4f7b-9acf-09f1a8035fed" containerName="test-curl-7462-ef7341" Sep 11 12:09:04.720448 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:04.720390 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a45ba51-03da-4f7b-9acf-09f1a8035fed" containerName="test-curl-7462-ef7341" Sep 11 12:09:04.720579 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:04.720471 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a45ba51-03da-4f7b-9acf-09f1a8035fed" containerName="test-curl-7462-ef7341" Sep 11 12:09:04.723501 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:04.723480 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" Sep 11 12:09:04.726166 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:09:04.726129 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7462-e81687\" is forbidden: User \"system:node:ip-10-0-132-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-170.ec2.internal' and this object" podUID="3d30c943-d018-4150-bc78-92cb92fbb4ac" pod="opendatahub/test-curl-7462-e81687" Sep 11 12:09:04.726733 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:04.726699 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-8d375e02-openshift-default-dockercfg-98dhw\"" Sep 11 12:09:04.738405 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:04.738378 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl"] Sep 11 12:09:04.759112 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:04.759079 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d30c943-d018-4150-bc78-92cb92fbb4ac" path="/var/lib/kubelet/pods/3d30c943-d018-4150-bc78-92cb92fbb4ac/volumes" Sep 11 12:09:04.891036 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:04.890993 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88f56cde-f00a-407c-a7cc-ba836c8a07e9-istio-podinfo\") pod \"e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl\" (UID: \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\") " pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" Sep 11 12:09:04.891225 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:04.891060 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88f56cde-f00a-407c-a7cc-ba836c8a07e9-workload-socket\") pod \"e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl\" (UID: \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\") " pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" Sep 11 12:09:04.891225 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:04.891170 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88f56cde-f00a-407c-a7cc-ba836c8a07e9-credential-socket\") pod \"e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl\" (UID: \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\") " pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" Sep 11 12:09:04.891225 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:04.891211 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88f56cde-f00a-407c-a7cc-ba836c8a07e9-istio-data\") pod \"e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl\" (UID: \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\") " pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" Sep 11 12:09:04.891432 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:04.891238 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88f56cde-f00a-407c-a7cc-ba836c8a07e9-workload-certs\") pod \"e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl\" (UID: \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\") " pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" Sep 11 12:09:04.891432 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:04.891268 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f97tg\" (UniqueName: \"kubernetes.io/projected/88f56cde-f00a-407c-a7cc-ba836c8a07e9-kube-api-access-f97tg\") pod \"e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl\" (UID: \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\") " pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" Sep 11 12:09:04.891432 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:04.891328 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88f56cde-f00a-407c-a7cc-ba836c8a07e9-istio-token\") pod \"e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl\" (UID: \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\") " pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" Sep 11 12:09:04.891432 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:04.891358 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88f56cde-f00a-407c-a7cc-ba836c8a07e9-istiod-ca-cert\") pod \"e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl\" (UID: \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\") " pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" Sep 11 12:09:04.891605 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:04.891448 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88f56cde-f00a-407c-a7cc-ba836c8a07e9-istio-envoy\") pod \"e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl\" (UID: \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\") " pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" Sep 11 12:09:04.992583 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:04.992485 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88f56cde-f00a-407c-a7cc-ba836c8a07e9-workload-socket\") pod \"e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl\" (UID: \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\") " pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" Sep 11 12:09:04.992583 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:04.992530 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88f56cde-f00a-407c-a7cc-ba836c8a07e9-credential-socket\") pod \"e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl\" (UID: \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\") " pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" Sep 11 12:09:04.992583 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:04.992552 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88f56cde-f00a-407c-a7cc-ba836c8a07e9-istio-data\") pod \"e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl\" (UID: \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\") " pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" Sep 11 12:09:04.992900 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:04.992668 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88f56cde-f00a-407c-a7cc-ba836c8a07e9-workload-certs\") pod \"e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl\" (UID: \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\") " pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" Sep 11 12:09:04.992900 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:04.992714 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f97tg\" (UniqueName: \"kubernetes.io/projected/88f56cde-f00a-407c-a7cc-ba836c8a07e9-kube-api-access-f97tg\") pod \"e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl\" (UID: \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\") " pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" Sep 11 12:09:04.992900 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:04.992765 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88f56cde-f00a-407c-a7cc-ba836c8a07e9-istio-token\") pod \"e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl\" (UID: \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\") " pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" Sep 11 12:09:04.992900 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:04.992802 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88f56cde-f00a-407c-a7cc-ba836c8a07e9-istiod-ca-cert\") pod \"e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl\" (UID: \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\") " pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" Sep 11 12:09:04.992900 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:04.992847 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88f56cde-f00a-407c-a7cc-ba836c8a07e9-istio-envoy\") pod \"e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl\" (UID: \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\") " pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" Sep 11 12:09:04.993178 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:04.992928 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88f56cde-f00a-407c-a7cc-ba836c8a07e9-istio-podinfo\") pod \"e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl\" (UID: \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\") " pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" Sep 11 12:09:04.993178 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:04.992959 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88f56cde-f00a-407c-a7cc-ba836c8a07e9-credential-socket\") pod \"e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl\" (UID: \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\") " pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" Sep 11 12:09:04.993178 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:04.992932 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88f56cde-f00a-407c-a7cc-ba836c8a07e9-istio-data\") pod \"e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl\" (UID: \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\") " pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" Sep 11 12:09:04.993178 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:04.993150 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88f56cde-f00a-407c-a7cc-ba836c8a07e9-workload-certs\") pod \"e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl\" (UID: \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\") " pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" Sep 11 12:09:04.993370 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:04.993311 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88f56cde-f00a-407c-a7cc-ba836c8a07e9-workload-socket\") pod \"e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl\" (UID: \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\") " pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" Sep 11 12:09:04.993579 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:04.993559 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88f56cde-f00a-407c-a7cc-ba836c8a07e9-istiod-ca-cert\") pod \"e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl\" (UID: \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\") " pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" Sep 11 12:09:04.995040 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:04.995019 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88f56cde-f00a-407c-a7cc-ba836c8a07e9-istio-envoy\") pod \"e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl\" (UID: \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\") " pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" Sep 11 12:09:04.995259 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:04.995242 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88f56cde-f00a-407c-a7cc-ba836c8a07e9-istio-podinfo\") pod \"e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl\" (UID: \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\") " pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" Sep 11 12:09:05.001841 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:05.001821 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88f56cde-f00a-407c-a7cc-ba836c8a07e9-istio-token\") pod \"e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl\" (UID: \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\") " pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" Sep 11 12:09:05.001841 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:05.001839 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f97tg\" (UniqueName: \"kubernetes.io/projected/88f56cde-f00a-407c-a7cc-ba836c8a07e9-kube-api-access-f97tg\") pod \"e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl\" (UID: \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\") " pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" Sep 11 12:09:05.034087 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:05.034052 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" Sep 11 12:09:05.282345 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:05.282315 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7462-ef7341" podUID="0a45ba51-03da-4f7b-9acf-09f1a8035fed" containerName="test-curl-7462-ef7341" containerID="cri-o://977c3088f2b22d81d54d6d2a455da3c20b5eb7c7810f1c7f009c07419dc98650" gracePeriod=2 Sep 11 12:09:05.326821 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:05.326797 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl"] Sep 11 12:09:05.341788 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:09:05.341745 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88f56cde_f00a_407c_a7cc_ba836c8a07e9.slice/crio-c90f732b0579a1882a565eb5bd3af879caa6f34b539758294cfb008a82c32d82 WatchSource:0}: Error finding container c90f732b0579a1882a565eb5bd3af879caa6f34b539758294cfb008a82c32d82: Status 404 returned error can't find the container with id c90f732b0579a1882a565eb5bd3af879caa6f34b539758294cfb008a82c32d82 Sep 11 12:09:05.344292 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:05.344249 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:09:05.344408 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:05.344345 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:09:05.344408 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:05.344391 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:09:05.428898 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:05.428849 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7462-e91c31" Sep 11 12:09:05.600344 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:05.600313 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gwb2p\" (UniqueName: \"kubernetes.io/projected/f68dbcdf-e52f-40e0-b5c6-35e7cac20379-kube-api-access-gwb2p\") pod \"f68dbcdf-e52f-40e0-b5c6-35e7cac20379\" (UID: \"f68dbcdf-e52f-40e0-b5c6-35e7cac20379\") " Sep 11 12:09:05.602450 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:05.602425 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f68dbcdf-e52f-40e0-b5c6-35e7cac20379-kube-api-access-gwb2p" (OuterVolumeSpecName: "kube-api-access-gwb2p") pod "f68dbcdf-e52f-40e0-b5c6-35e7cac20379" (UID: "f68dbcdf-e52f-40e0-b5c6-35e7cac20379"). InnerVolumeSpecName "kube-api-access-gwb2p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:09:05.701647 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:05.701608 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwb2p\" (UniqueName: \"kubernetes.io/projected/f68dbcdf-e52f-40e0-b5c6-35e7cac20379-kube-api-access-gwb2p\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:09:06.288918 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:06.288805 2566 generic.go:356] "Generic (PLEG): container finished" podID="f68dbcdf-e52f-40e0-b5c6-35e7cac20379" containerID="5e3fe7a2aa0352d8244a1b848325b221c165af8086f915bb3022bf325402b62d" exitCode=137 Sep 11 12:09:06.288918 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:06.288851 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7462-e91c31" Sep 11 12:09:06.288918 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:06.288906 2566 scope.go:117] "RemoveContainer" containerID="5e3fe7a2aa0352d8244a1b848325b221c165af8086f915bb3022bf325402b62d" Sep 11 12:09:06.290551 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:06.290526 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" event={"ID":"88f56cde-f00a-407c-a7cc-ba836c8a07e9","Type":"ContainerStarted","Data":"448fc2c7864ecc94a7fade9ad632f0cb9f57955f7d1ffc6773c3b1f721368444"} Sep 11 12:09:06.290658 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:06.290581 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" event={"ID":"88f56cde-f00a-407c-a7cc-ba836c8a07e9","Type":"ContainerStarted","Data":"c90f732b0579a1882a565eb5bd3af879caa6f34b539758294cfb008a82c32d82"} Sep 11 12:09:06.298299 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:06.298283 2566 scope.go:117] "RemoveContainer" containerID="5e3fe7a2aa0352d8244a1b848325b221c165af8086f915bb3022bf325402b62d" Sep 11 12:09:06.298606 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:09:06.298589 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e3fe7a2aa0352d8244a1b848325b221c165af8086f915bb3022bf325402b62d\": container with ID starting with 5e3fe7a2aa0352d8244a1b848325b221c165af8086f915bb3022bf325402b62d not found: ID does not exist" containerID="5e3fe7a2aa0352d8244a1b848325b221c165af8086f915bb3022bf325402b62d" Sep 11 12:09:06.298681 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:06.298617 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e3fe7a2aa0352d8244a1b848325b221c165af8086f915bb3022bf325402b62d"} err="failed to get container status \"5e3fe7a2aa0352d8244a1b848325b221c165af8086f915bb3022bf325402b62d\": rpc error: code = NotFound desc = could not find container \"5e3fe7a2aa0352d8244a1b848325b221c165af8086f915bb3022bf325402b62d\": container with ID starting with 5e3fe7a2aa0352d8244a1b848325b221c165af8086f915bb3022bf325402b62d not found: ID does not exist" Sep 11 12:09:06.319563 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:09:06.319523 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7462-e91c31\" is forbidden: User \"system:node:ip-10-0-132-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-170.ec2.internal' and this object" podUID="f68dbcdf-e52f-40e0-b5c6-35e7cac20379" pod="opendatahub/test-curl-7462-e91c31" Sep 11 12:09:06.758777 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:06.758744 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f68dbcdf-e52f-40e0-b5c6-35e7cac20379" path="/var/lib/kubelet/pods/f68dbcdf-e52f-40e0-b5c6-35e7cac20379/volumes" Sep 11 12:09:07.034477 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:07.034387 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" Sep 11 12:09:07.040008 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:07.039979 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" Sep 11 12:09:07.069700 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:07.069647 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" podStartSLOduration=3.069630445 podStartE2EDuration="3.069630445s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:09:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:09:06.317577633 +0000 UTC m=+1278.082550811" watchObservedRunningTime="2026-09-11 12:09:07.069630445 +0000 UTC m=+1278.834603620" Sep 11 12:09:07.297001 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:07.296967 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" Sep 11 12:09:07.297881 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:07.297850 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" Sep 11 12:09:07.462892 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:07.462848 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7462-ef7341" Sep 11 12:09:07.618175 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:07.618135 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-652f8\" (UniqueName: \"kubernetes.io/projected/0a45ba51-03da-4f7b-9acf-09f1a8035fed-kube-api-access-652f8\") pod \"0a45ba51-03da-4f7b-9acf-09f1a8035fed\" (UID: \"0a45ba51-03da-4f7b-9acf-09f1a8035fed\") " Sep 11 12:09:07.620199 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:07.620172 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a45ba51-03da-4f7b-9acf-09f1a8035fed-kube-api-access-652f8" (OuterVolumeSpecName: "kube-api-access-652f8") pod "0a45ba51-03da-4f7b-9acf-09f1a8035fed" (UID: "0a45ba51-03da-4f7b-9acf-09f1a8035fed"). InnerVolumeSpecName "kube-api-access-652f8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:09:07.719718 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:07.719685 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-652f8\" (UniqueName: \"kubernetes.io/projected/0a45ba51-03da-4f7b-9acf-09f1a8035fed-kube-api-access-652f8\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:09:08.303455 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:08.303418 2566 generic.go:356] "Generic (PLEG): container finished" podID="0a45ba51-03da-4f7b-9acf-09f1a8035fed" containerID="977c3088f2b22d81d54d6d2a455da3c20b5eb7c7810f1c7f009c07419dc98650" exitCode=137 Sep 11 12:09:08.303966 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:08.303483 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7462-ef7341" Sep 11 12:09:08.303966 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:08.303524 2566 scope.go:117] "RemoveContainer" containerID="977c3088f2b22d81d54d6d2a455da3c20b5eb7c7810f1c7f009c07419dc98650" Sep 11 12:09:08.318563 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:09:08.318525 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7462-ef7341\" is forbidden: User \"system:node:ip-10-0-132-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-170.ec2.internal' and this object" podUID="0a45ba51-03da-4f7b-9acf-09f1a8035fed" pod="opendatahub/test-curl-7462-ef7341" Sep 11 12:09:08.321296 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:08.321261 2566 scope.go:117] "RemoveContainer" containerID="977c3088f2b22d81d54d6d2a455da3c20b5eb7c7810f1c7f009c07419dc98650" Sep 11 12:09:08.321727 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:09:08.321706 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"977c3088f2b22d81d54d6d2a455da3c20b5eb7c7810f1c7f009c07419dc98650\": container with ID starting with 977c3088f2b22d81d54d6d2a455da3c20b5eb7c7810f1c7f009c07419dc98650 not found: ID does not exist" containerID="977c3088f2b22d81d54d6d2a455da3c20b5eb7c7810f1c7f009c07419dc98650" Sep 11 12:09:08.321785 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:08.321736 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"977c3088f2b22d81d54d6d2a455da3c20b5eb7c7810f1c7f009c07419dc98650"} err="failed to get container status \"977c3088f2b22d81d54d6d2a455da3c20b5eb7c7810f1c7f009c07419dc98650\": rpc error: code = NotFound desc = could not find container \"977c3088f2b22d81d54d6d2a455da3c20b5eb7c7810f1c7f009c07419dc98650\": container with ID starting with 977c3088f2b22d81d54d6d2a455da3c20b5eb7c7810f1c7f009c07419dc98650 not found: ID does not exist" Sep 11 12:09:08.757329 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:09:08.757290 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7462-ef7341\" is forbidden: User \"system:node:ip-10-0-132-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-170.ec2.internal' and this object" podUID="0a45ba51-03da-4f7b-9acf-09f1a8035fed" pod="opendatahub/test-curl-7462-ef7341" Sep 11 12:09:08.758767 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:08.758740 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a45ba51-03da-4f7b-9acf-09f1a8035fed" path="/var/lib/kubelet/pods/0a45ba51-03da-4f7b-9acf-09f1a8035fed/volumes" Sep 11 12:09:10.324101 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:10.324067 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8d375e02-79c6f95b97-bdbf2"] Sep 11 12:09:10.327323 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:10.327298 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-8d375e02-79c6f95b97-bdbf2" Sep 11 12:09:10.330140 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:10.330117 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-8d375e02\"" Sep 11 12:09:10.330276 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:10.330118 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-8d375e02-dockercfg-p7sf2\"" Sep 11 12:09:10.339836 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:10.339811 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8d375e02-79c6f95b97-bdbf2"] Sep 11 12:09:10.443990 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:10.443956 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e54810a3-c9d3-4fbf-b38e-36250f5454c1-plugins-config-volume\") pod \"payload-processing-e2e-mt-8d375e02-79c6f95b97-bdbf2\" (UID: \"e54810a3-c9d3-4fbf-b38e-36250f5454c1\") " pod="openshift-ingress/payload-processing-e2e-mt-8d375e02-79c6f95b97-bdbf2" Sep 11 12:09:10.444150 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:10.444070 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c95c7\" (UniqueName: \"kubernetes.io/projected/e54810a3-c9d3-4fbf-b38e-36250f5454c1-kube-api-access-c95c7\") pod \"payload-processing-e2e-mt-8d375e02-79c6f95b97-bdbf2\" (UID: \"e54810a3-c9d3-4fbf-b38e-36250f5454c1\") " pod="openshift-ingress/payload-processing-e2e-mt-8d375e02-79c6f95b97-bdbf2" Sep 11 12:09:10.544767 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:10.544728 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c95c7\" (UniqueName: \"kubernetes.io/projected/e54810a3-c9d3-4fbf-b38e-36250f5454c1-kube-api-access-c95c7\") pod \"payload-processing-e2e-mt-8d375e02-79c6f95b97-bdbf2\" (UID: \"e54810a3-c9d3-4fbf-b38e-36250f5454c1\") " pod="openshift-ingress/payload-processing-e2e-mt-8d375e02-79c6f95b97-bdbf2" Sep 11 12:09:10.544933 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:10.544798 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e54810a3-c9d3-4fbf-b38e-36250f5454c1-plugins-config-volume\") pod \"payload-processing-e2e-mt-8d375e02-79c6f95b97-bdbf2\" (UID: \"e54810a3-c9d3-4fbf-b38e-36250f5454c1\") " pod="openshift-ingress/payload-processing-e2e-mt-8d375e02-79c6f95b97-bdbf2" Sep 11 12:09:10.545512 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:10.545493 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e54810a3-c9d3-4fbf-b38e-36250f5454c1-plugins-config-volume\") pod \"payload-processing-e2e-mt-8d375e02-79c6f95b97-bdbf2\" (UID: \"e54810a3-c9d3-4fbf-b38e-36250f5454c1\") " pod="openshift-ingress/payload-processing-e2e-mt-8d375e02-79c6f95b97-bdbf2" Sep 11 12:09:10.553572 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:10.553546 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c95c7\" (UniqueName: \"kubernetes.io/projected/e54810a3-c9d3-4fbf-b38e-36250f5454c1-kube-api-access-c95c7\") pod \"payload-processing-e2e-mt-8d375e02-79c6f95b97-bdbf2\" (UID: \"e54810a3-c9d3-4fbf-b38e-36250f5454c1\") " pod="openshift-ingress/payload-processing-e2e-mt-8d375e02-79c6f95b97-bdbf2" Sep 11 12:09:10.637313 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:10.637238 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-8d375e02-79c6f95b97-bdbf2" Sep 11 12:09:10.910521 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:10.910363 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8d375e02-79c6f95b97-bdbf2"] Sep 11 12:09:10.913359 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:09:10.913325 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode54810a3_c9d3_4fbf_b38e_36250f5454c1.slice/crio-ae8a9cd239741b358966f164e6903d563bf2906583185524c5d973b0a7a70840 WatchSource:0}: Error finding container ae8a9cd239741b358966f164e6903d563bf2906583185524c5d973b0a7a70840: Status 404 returned error can't find the container with id ae8a9cd239741b358966f164e6903d563bf2906583185524c5d973b0a7a70840 Sep 11 12:09:11.324256 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:11.324216 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-8d375e02-79c6f95b97-bdbf2" event={"ID":"e54810a3-c9d3-4fbf-b38e-36250f5454c1","Type":"ContainerStarted","Data":"38e89cc206cafffb15bd5e674f1ff56f2ff943259d0de91b9027f854b436987f"} Sep 11 12:09:11.324752 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:11.324272 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-8d375e02-79c6f95b97-bdbf2" event={"ID":"e54810a3-c9d3-4fbf-b38e-36250f5454c1","Type":"ContainerStarted","Data":"ae8a9cd239741b358966f164e6903d563bf2906583185524c5d973b0a7a70840"} Sep 11 12:09:11.324752 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:11.324301 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-8d375e02-79c6f95b97-bdbf2" Sep 11 12:09:11.343081 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:11.343017 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-8d375e02-79c6f95b97-bdbf2" podStartSLOduration=1.342994763 podStartE2EDuration="1.342994763s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:09:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:09:11.342185548 +0000 UTC m=+1283.107158724" watchObservedRunningTime="2026-09-11 12:09:11.342994763 +0000 UTC m=+1283.107967941" Sep 11 12:09:22.331784 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:22.331751 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-8d375e02-79c6f95b97-bdbf2" Sep 11 12:09:28.413267 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:28.413187 2566 generic.go:356] "Generic (PLEG): container finished" podID="88f56cde-f00a-407c-a7cc-ba836c8a07e9" containerID="448fc2c7864ecc94a7fade9ad632f0cb9f57955f7d1ffc6773c3b1f721368444" exitCode=0 Sep 11 12:09:28.413267 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:28.413228 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" event={"ID":"88f56cde-f00a-407c-a7cc-ba836c8a07e9","Type":"ContainerDied","Data":"448fc2c7864ecc94a7fade9ad632f0cb9f57955f7d1ffc6773c3b1f721368444"} Sep 11 12:09:28.413692 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:28.413632 2566 scope.go:117] "RemoveContainer" containerID="448fc2c7864ecc94a7fade9ad632f0cb9f57955f7d1ffc6773c3b1f721368444" Sep 11 12:09:28.414672 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:28.414641 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:09:28.414731 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:28.414717 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:09:28.414767 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:28.414748 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:09:29.425672 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:29.425633 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" event={"ID":"88f56cde-f00a-407c-a7cc-ba836c8a07e9","Type":"ContainerStarted","Data":"28b5ec63d12334eafe88d2da0aa88b20edb27bab4cb6c17508f86b43adbc56aa"} Sep 11 12:09:30.034953 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:30.034919 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" Sep 11 12:09:30.035161 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:30.035135 2566 patch_prober.go:28] interesting pod/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Sep 11 12:09:30.035212 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:30.035191 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" podUID="88f56cde-f00a-407c-a7cc-ba836c8a07e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Sep 11 12:09:31.034655 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:31.034611 2566 patch_prober.go:28] interesting pod/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Sep 11 12:09:31.035082 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:31.034680 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" podUID="88f56cde-f00a-407c-a7cc-ba836c8a07e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Sep 11 12:09:32.035223 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:32.035183 2566 patch_prober.go:28] interesting pod/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Sep 11 12:09:32.035626 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:32.035240 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" podUID="88f56cde-f00a-407c-a7cc-ba836c8a07e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Sep 11 12:09:33.035239 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:33.035201 2566 patch_prober.go:28] interesting pod/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Sep 11 12:09:33.035642 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:33.035262 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" podUID="88f56cde-f00a-407c-a7cc-ba836c8a07e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Sep 11 12:09:34.035437 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:34.035397 2566 patch_prober.go:28] interesting pod/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Sep 11 12:09:34.035832 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:34.035455 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" podUID="88f56cde-f00a-407c-a7cc-ba836c8a07e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Sep 11 12:09:35.039022 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:35.038988 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" Sep 11 12:09:35.039483 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:35.039266 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" Sep 11 12:09:35.040018 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:09:35.040000 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" Sep 11 12:10:38.296588 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:10:38.296553 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8d375e02-79c6f95b97-bdbf2"] Sep 11 12:10:38.297074 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:10:38.296836 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-8d375e02-79c6f95b97-bdbf2" podUID="e54810a3-c9d3-4fbf-b38e-36250f5454c1" containerName="payload-processing" containerID="cri-o://38e89cc206cafffb15bd5e674f1ff56f2ff943259d0de91b9027f854b436987f" gracePeriod=30 Sep 11 12:10:38.541742 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:10:38.541716 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-8d375e02-79c6f95b97-bdbf2" Sep 11 12:10:38.657719 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:10:38.657685 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e54810a3-c9d3-4fbf-b38e-36250f5454c1-plugins-config-volume\") pod \"e54810a3-c9d3-4fbf-b38e-36250f5454c1\" (UID: \"e54810a3-c9d3-4fbf-b38e-36250f5454c1\") " Sep 11 12:10:38.657908 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:10:38.657742 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c95c7\" (UniqueName: \"kubernetes.io/projected/e54810a3-c9d3-4fbf-b38e-36250f5454c1-kube-api-access-c95c7\") pod \"e54810a3-c9d3-4fbf-b38e-36250f5454c1\" (UID: \"e54810a3-c9d3-4fbf-b38e-36250f5454c1\") " Sep 11 12:10:38.658108 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:10:38.658084 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e54810a3-c9d3-4fbf-b38e-36250f5454c1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e54810a3-c9d3-4fbf-b38e-36250f5454c1" (UID: "e54810a3-c9d3-4fbf-b38e-36250f5454c1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 12:10:38.659914 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:10:38.659885 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e54810a3-c9d3-4fbf-b38e-36250f5454c1-kube-api-access-c95c7" (OuterVolumeSpecName: "kube-api-access-c95c7") pod "e54810a3-c9d3-4fbf-b38e-36250f5454c1" (UID: "e54810a3-c9d3-4fbf-b38e-36250f5454c1"). InnerVolumeSpecName "kube-api-access-c95c7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:10:38.758527 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:10:38.758485 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e54810a3-c9d3-4fbf-b38e-36250f5454c1-plugins-config-volume\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:10:38.758527 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:10:38.758510 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c95c7\" (UniqueName: \"kubernetes.io/projected/e54810a3-c9d3-4fbf-b38e-36250f5454c1-kube-api-access-c95c7\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:10:38.771623 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:10:38.771595 2566 generic.go:356] "Generic (PLEG): container finished" podID="e54810a3-c9d3-4fbf-b38e-36250f5454c1" containerID="38e89cc206cafffb15bd5e674f1ff56f2ff943259d0de91b9027f854b436987f" exitCode=0 Sep 11 12:10:38.771736 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:10:38.771637 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-8d375e02-79c6f95b97-bdbf2" Sep 11 12:10:38.771736 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:10:38.771672 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-8d375e02-79c6f95b97-bdbf2" event={"ID":"e54810a3-c9d3-4fbf-b38e-36250f5454c1","Type":"ContainerDied","Data":"38e89cc206cafffb15bd5e674f1ff56f2ff943259d0de91b9027f854b436987f"} Sep 11 12:10:38.771736 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:10:38.771723 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-8d375e02-79c6f95b97-bdbf2" event={"ID":"e54810a3-c9d3-4fbf-b38e-36250f5454c1","Type":"ContainerDied","Data":"ae8a9cd239741b358966f164e6903d563bf2906583185524c5d973b0a7a70840"} Sep 11 12:10:38.771880 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:10:38.771749 2566 scope.go:117] "RemoveContainer" containerID="38e89cc206cafffb15bd5e674f1ff56f2ff943259d0de91b9027f854b436987f" Sep 11 12:10:38.782327 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:10:38.782309 2566 scope.go:117] "RemoveContainer" containerID="38e89cc206cafffb15bd5e674f1ff56f2ff943259d0de91b9027f854b436987f" Sep 11 12:10:38.782617 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:10:38.782594 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38e89cc206cafffb15bd5e674f1ff56f2ff943259d0de91b9027f854b436987f\": container with ID starting with 38e89cc206cafffb15bd5e674f1ff56f2ff943259d0de91b9027f854b436987f not found: ID does not exist" containerID="38e89cc206cafffb15bd5e674f1ff56f2ff943259d0de91b9027f854b436987f" Sep 11 12:10:38.782666 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:10:38.782627 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38e89cc206cafffb15bd5e674f1ff56f2ff943259d0de91b9027f854b436987f"} err="failed to get container status \"38e89cc206cafffb15bd5e674f1ff56f2ff943259d0de91b9027f854b436987f\": rpc error: code = NotFound desc = could not find container \"38e89cc206cafffb15bd5e674f1ff56f2ff943259d0de91b9027f854b436987f\": container with ID starting with 38e89cc206cafffb15bd5e674f1ff56f2ff943259d0de91b9027f854b436987f not found: ID does not exist" Sep 11 12:10:38.790720 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:10:38.790697 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8d375e02-79c6f95b97-bdbf2"] Sep 11 12:10:38.796003 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:10:38.795984 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8d375e02-79c6f95b97-bdbf2"] Sep 11 12:10:40.758652 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:10:40.758620 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e54810a3-c9d3-4fbf-b38e-36250f5454c1" path="/var/lib/kubelet/pods/e54810a3-c9d3-4fbf-b38e-36250f5454c1/volumes" Sep 11 12:11:18.152595 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:18.152561 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7465-42f5e8"] Sep 11 12:11:18.153104 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:18.152946 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e54810a3-c9d3-4fbf-b38e-36250f5454c1" containerName="payload-processing" Sep 11 12:11:18.153104 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:18.152959 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e54810a3-c9d3-4fbf-b38e-36250f5454c1" containerName="payload-processing" Sep 11 12:11:18.153104 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:18.153023 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e54810a3-c9d3-4fbf-b38e-36250f5454c1" containerName="payload-processing" Sep 11 12:11:18.155626 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:18.155604 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-42f5e8" Sep 11 12:11:18.158500 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:18.158477 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-wkgmv\"" Sep 11 12:11:18.163392 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:18.163366 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7465-42f5e8"] Sep 11 12:11:18.235787 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:18.235753 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nfcn\" (UniqueName: \"kubernetes.io/projected/7843f46f-8a48-4a5a-8e8d-426a8e858b3a-kube-api-access-6nfcn\") pod \"test-curl-7465-42f5e8\" (UID: \"7843f46f-8a48-4a5a-8e8d-426a8e858b3a\") " pod="opendatahub/test-curl-7465-42f5e8" Sep 11 12:11:18.337178 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:18.337143 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nfcn\" (UniqueName: \"kubernetes.io/projected/7843f46f-8a48-4a5a-8e8d-426a8e858b3a-kube-api-access-6nfcn\") pod \"test-curl-7465-42f5e8\" (UID: \"7843f46f-8a48-4a5a-8e8d-426a8e858b3a\") " pod="opendatahub/test-curl-7465-42f5e8" Sep 11 12:11:18.345675 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:18.345647 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nfcn\" (UniqueName: \"kubernetes.io/projected/7843f46f-8a48-4a5a-8e8d-426a8e858b3a-kube-api-access-6nfcn\") pod \"test-curl-7465-42f5e8\" (UID: \"7843f46f-8a48-4a5a-8e8d-426a8e858b3a\") " pod="opendatahub/test-curl-7465-42f5e8" Sep 11 12:11:18.465759 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:18.465664 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-42f5e8" Sep 11 12:11:18.740646 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:18.740617 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7465-42f5e8"] Sep 11 12:11:18.742262 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:11:18.742239 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7843f46f_8a48_4a5a_8e8d_426a8e858b3a.slice/crio-2f2426923afbb49ddb96b728abb89cd921766770202c64852ace697ccf1a6595 WatchSource:0}: Error finding container 2f2426923afbb49ddb96b728abb89cd921766770202c64852ace697ccf1a6595: Status 404 returned error can't find the container with id 2f2426923afbb49ddb96b728abb89cd921766770202c64852ace697ccf1a6595 Sep 11 12:11:18.970460 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:18.970428 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7465-42f5e8" event={"ID":"7843f46f-8a48-4a5a-8e8d-426a8e858b3a","Type":"ContainerStarted","Data":"5ca6fa61d61a781bbf370de46a4f8a64d34f3b03305fef2a9cf865c015b945a5"} Sep 11 12:11:18.970632 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:18.970468 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7465-42f5e8" event={"ID":"7843f46f-8a48-4a5a-8e8d-426a8e858b3a","Type":"ContainerStarted","Data":"2f2426923afbb49ddb96b728abb89cd921766770202c64852ace697ccf1a6595"} Sep 11 12:11:18.990563 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:18.990475 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7465-42f5e8" podStartSLOduration=0.99045956 podStartE2EDuration="990.45956ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:11:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:11:18.988502198 +0000 UTC m=+1410.753475373" watchObservedRunningTime="2026-09-11 12:11:18.99045956 +0000 UTC m=+1410.755432735" Sep 11 12:11:19.610384 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:19.610349 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7465-42f5e8"] Sep 11 12:11:19.612988 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:19.612963 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7465-42f5e8"] Sep 11 12:11:19.977731 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:11:19.977697 2566 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7465-42f5e8" container="test-curl-7465-42f5e8" Sep 11 12:11:20.981681 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:20.981637 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7465-42f5e8" podUID="7843f46f-8a48-4a5a-8e8d-426a8e858b3a" containerName="test-curl-7465-42f5e8" containerID="cri-o://5ca6fa61d61a781bbf370de46a4f8a64d34f3b03305fef2a9cf865c015b945a5" gracePeriod=2 Sep 11 12:11:23.156759 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:23.156734 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-42f5e8" Sep 11 12:11:23.280305 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:23.280216 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6nfcn\" (UniqueName: \"kubernetes.io/projected/7843f46f-8a48-4a5a-8e8d-426a8e858b3a-kube-api-access-6nfcn\") pod \"7843f46f-8a48-4a5a-8e8d-426a8e858b3a\" (UID: \"7843f46f-8a48-4a5a-8e8d-426a8e858b3a\") " Sep 11 12:11:23.282239 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:23.282214 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7843f46f-8a48-4a5a-8e8d-426a8e858b3a-kube-api-access-6nfcn" (OuterVolumeSpecName: "kube-api-access-6nfcn") pod "7843f46f-8a48-4a5a-8e8d-426a8e858b3a" (UID: "7843f46f-8a48-4a5a-8e8d-426a8e858b3a"). InnerVolumeSpecName "kube-api-access-6nfcn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:11:23.381331 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:23.381299 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6nfcn\" (UniqueName: \"kubernetes.io/projected/7843f46f-8a48-4a5a-8e8d-426a8e858b3a-kube-api-access-6nfcn\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:11:23.997745 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:23.997707 2566 generic.go:356] "Generic (PLEG): container finished" podID="7843f46f-8a48-4a5a-8e8d-426a8e858b3a" containerID="5ca6fa61d61a781bbf370de46a4f8a64d34f3b03305fef2a9cf865c015b945a5" exitCode=137 Sep 11 12:11:23.997745 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:23.997749 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-42f5e8" Sep 11 12:11:23.998026 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:23.997775 2566 scope.go:117] "RemoveContainer" containerID="5ca6fa61d61a781bbf370de46a4f8a64d34f3b03305fef2a9cf865c015b945a5" Sep 11 12:11:24.006688 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:24.006666 2566 scope.go:117] "RemoveContainer" containerID="5ca6fa61d61a781bbf370de46a4f8a64d34f3b03305fef2a9cf865c015b945a5" Sep 11 12:11:24.007067 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:11:24.007036 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ca6fa61d61a781bbf370de46a4f8a64d34f3b03305fef2a9cf865c015b945a5\": container with ID starting with 5ca6fa61d61a781bbf370de46a4f8a64d34f3b03305fef2a9cf865c015b945a5 not found: ID does not exist" containerID="5ca6fa61d61a781bbf370de46a4f8a64d34f3b03305fef2a9cf865c015b945a5" Sep 11 12:11:24.007166 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:24.007083 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ca6fa61d61a781bbf370de46a4f8a64d34f3b03305fef2a9cf865c015b945a5"} err="failed to get container status \"5ca6fa61d61a781bbf370de46a4f8a64d34f3b03305fef2a9cf865c015b945a5\": rpc error: code = NotFound desc = could not find container \"5ca6fa61d61a781bbf370de46a4f8a64d34f3b03305fef2a9cf865c015b945a5\": container with ID starting with 5ca6fa61d61a781bbf370de46a4f8a64d34f3b03305fef2a9cf865c015b945a5 not found: ID does not exist" Sep 11 12:11:24.008323 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:11:24.008293 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7465-42f5e8\" is forbidden: User \"system:node:ip-10-0-132-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-170.ec2.internal' and this object" podUID="7843f46f-8a48-4a5a-8e8d-426a8e858b3a" pod="opendatahub/test-curl-7465-42f5e8" Sep 11 12:11:24.759693 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:24.759661 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7843f46f-8a48-4a5a-8e8d-426a8e858b3a" path="/var/lib/kubelet/pods/7843f46f-8a48-4a5a-8e8d-426a8e858b3a/volumes" Sep 11 12:11:27.649095 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:27.649057 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-af081c/sub-test-model-af081c-kserve-6fc55f5c89-kcpwb"] Sep 11 12:11:27.649489 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:27.649474 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7843f46f-8a48-4a5a-8e8d-426a8e858b3a" containerName="test-curl-7465-42f5e8" Sep 11 12:11:27.649541 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:27.649493 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7843f46f-8a48-4a5a-8e8d-426a8e858b3a" containerName="test-curl-7465-42f5e8" Sep 11 12:11:27.649581 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:27.649570 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7843f46f-8a48-4a5a-8e8d-426a8e858b3a" containerName="test-curl-7465-42f5e8" Sep 11 12:11:27.653937 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:27.653918 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-af081c/sub-test-model-af081c-kserve-6fc55f5c89-kcpwb" Sep 11 12:11:27.657651 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:27.657628 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-af081c\"/\"kube-root-ca.crt\"" Sep 11 12:11:27.657769 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:27.657715 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-af081c\"/\"sub-test-model-af081c-kserve-self-signed-certs\"" Sep 11 12:11:27.657769 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:27.657724 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-af081c\"/\"default-dockercfg-qbg9d\"" Sep 11 12:11:27.657908 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:27.657767 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-af081c\"/\"openshift-service-ca.crt\"" Sep 11 12:11:27.663913 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:27.663891 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-af081c/sub-test-model-af081c-kserve-6fc55f5c89-kcpwb"] Sep 11 12:11:27.724043 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:27.724009 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fh2vp\" (UniqueName: \"kubernetes.io/projected/e3aea040-d014-4057-93fd-f57019a37dce-kube-api-access-fh2vp\") pod \"sub-test-model-af081c-kserve-6fc55f5c89-kcpwb\" (UID: \"e3aea040-d014-4057-93fd-f57019a37dce\") " pod="ai-tenant-e2e-shared-a-w3-af081c/sub-test-model-af081c-kserve-6fc55f5c89-kcpwb" Sep 11 12:11:27.724210 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:27.724047 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3aea040-d014-4057-93fd-f57019a37dce-dshm\") pod \"sub-test-model-af081c-kserve-6fc55f5c89-kcpwb\" (UID: \"e3aea040-d014-4057-93fd-f57019a37dce\") " pod="ai-tenant-e2e-shared-a-w3-af081c/sub-test-model-af081c-kserve-6fc55f5c89-kcpwb" Sep 11 12:11:27.724210 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:27.724087 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3aea040-d014-4057-93fd-f57019a37dce-tmp-dir\") pod \"sub-test-model-af081c-kserve-6fc55f5c89-kcpwb\" (UID: \"e3aea040-d014-4057-93fd-f57019a37dce\") " pod="ai-tenant-e2e-shared-a-w3-af081c/sub-test-model-af081c-kserve-6fc55f5c89-kcpwb" Sep 11 12:11:27.724210 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:27.724158 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3aea040-d014-4057-93fd-f57019a37dce-home\") pod \"sub-test-model-af081c-kserve-6fc55f5c89-kcpwb\" (UID: \"e3aea040-d014-4057-93fd-f57019a37dce\") " pod="ai-tenant-e2e-shared-a-w3-af081c/sub-test-model-af081c-kserve-6fc55f5c89-kcpwb" Sep 11 12:11:27.724343 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:27.724240 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3aea040-d014-4057-93fd-f57019a37dce-model-cache\") pod \"sub-test-model-af081c-kserve-6fc55f5c89-kcpwb\" (UID: \"e3aea040-d014-4057-93fd-f57019a37dce\") " pod="ai-tenant-e2e-shared-a-w3-af081c/sub-test-model-af081c-kserve-6fc55f5c89-kcpwb" Sep 11 12:11:27.724343 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:27.724262 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3aea040-d014-4057-93fd-f57019a37dce-tls-certs\") pod \"sub-test-model-af081c-kserve-6fc55f5c89-kcpwb\" (UID: \"e3aea040-d014-4057-93fd-f57019a37dce\") " pod="ai-tenant-e2e-shared-a-w3-af081c/sub-test-model-af081c-kserve-6fc55f5c89-kcpwb" Sep 11 12:11:27.825401 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:27.825359 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3aea040-d014-4057-93fd-f57019a37dce-model-cache\") pod \"sub-test-model-af081c-kserve-6fc55f5c89-kcpwb\" (UID: \"e3aea040-d014-4057-93fd-f57019a37dce\") " pod="ai-tenant-e2e-shared-a-w3-af081c/sub-test-model-af081c-kserve-6fc55f5c89-kcpwb" Sep 11 12:11:27.825401 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:27.825400 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3aea040-d014-4057-93fd-f57019a37dce-tls-certs\") pod \"sub-test-model-af081c-kserve-6fc55f5c89-kcpwb\" (UID: \"e3aea040-d014-4057-93fd-f57019a37dce\") " pod="ai-tenant-e2e-shared-a-w3-af081c/sub-test-model-af081c-kserve-6fc55f5c89-kcpwb" Sep 11 12:11:27.825672 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:27.825525 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fh2vp\" (UniqueName: \"kubernetes.io/projected/e3aea040-d014-4057-93fd-f57019a37dce-kube-api-access-fh2vp\") pod \"sub-test-model-af081c-kserve-6fc55f5c89-kcpwb\" (UID: \"e3aea040-d014-4057-93fd-f57019a37dce\") " pod="ai-tenant-e2e-shared-a-w3-af081c/sub-test-model-af081c-kserve-6fc55f5c89-kcpwb" Sep 11 12:11:27.825672 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:27.825553 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3aea040-d014-4057-93fd-f57019a37dce-dshm\") pod \"sub-test-model-af081c-kserve-6fc55f5c89-kcpwb\" (UID: \"e3aea040-d014-4057-93fd-f57019a37dce\") " pod="ai-tenant-e2e-shared-a-w3-af081c/sub-test-model-af081c-kserve-6fc55f5c89-kcpwb" Sep 11 12:11:27.825672 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:27.825618 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3aea040-d014-4057-93fd-f57019a37dce-tmp-dir\") pod \"sub-test-model-af081c-kserve-6fc55f5c89-kcpwb\" (UID: \"e3aea040-d014-4057-93fd-f57019a37dce\") " pod="ai-tenant-e2e-shared-a-w3-af081c/sub-test-model-af081c-kserve-6fc55f5c89-kcpwb" Sep 11 12:11:27.825672 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:27.825666 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3aea040-d014-4057-93fd-f57019a37dce-home\") pod \"sub-test-model-af081c-kserve-6fc55f5c89-kcpwb\" (UID: \"e3aea040-d014-4057-93fd-f57019a37dce\") " pod="ai-tenant-e2e-shared-a-w3-af081c/sub-test-model-af081c-kserve-6fc55f5c89-kcpwb" Sep 11 12:11:27.825950 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:27.825797 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3aea040-d014-4057-93fd-f57019a37dce-model-cache\") pod \"sub-test-model-af081c-kserve-6fc55f5c89-kcpwb\" (UID: \"e3aea040-d014-4057-93fd-f57019a37dce\") " pod="ai-tenant-e2e-shared-a-w3-af081c/sub-test-model-af081c-kserve-6fc55f5c89-kcpwb" Sep 11 12:11:27.826159 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:27.826126 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3aea040-d014-4057-93fd-f57019a37dce-home\") pod \"sub-test-model-af081c-kserve-6fc55f5c89-kcpwb\" (UID: \"e3aea040-d014-4057-93fd-f57019a37dce\") " pod="ai-tenant-e2e-shared-a-w3-af081c/sub-test-model-af081c-kserve-6fc55f5c89-kcpwb" Sep 11 12:11:27.826159 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:27.826156 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3aea040-d014-4057-93fd-f57019a37dce-tmp-dir\") pod \"sub-test-model-af081c-kserve-6fc55f5c89-kcpwb\" (UID: \"e3aea040-d014-4057-93fd-f57019a37dce\") " pod="ai-tenant-e2e-shared-a-w3-af081c/sub-test-model-af081c-kserve-6fc55f5c89-kcpwb" Sep 11 12:11:27.827918 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:27.827900 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3aea040-d014-4057-93fd-f57019a37dce-dshm\") pod \"sub-test-model-af081c-kserve-6fc55f5c89-kcpwb\" (UID: \"e3aea040-d014-4057-93fd-f57019a37dce\") " pod="ai-tenant-e2e-shared-a-w3-af081c/sub-test-model-af081c-kserve-6fc55f5c89-kcpwb" Sep 11 12:11:27.828211 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:27.828192 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3aea040-d014-4057-93fd-f57019a37dce-tls-certs\") pod \"sub-test-model-af081c-kserve-6fc55f5c89-kcpwb\" (UID: \"e3aea040-d014-4057-93fd-f57019a37dce\") " pod="ai-tenant-e2e-shared-a-w3-af081c/sub-test-model-af081c-kserve-6fc55f5c89-kcpwb" Sep 11 12:11:27.834378 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:27.834353 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fh2vp\" (UniqueName: \"kubernetes.io/projected/e3aea040-d014-4057-93fd-f57019a37dce-kube-api-access-fh2vp\") pod \"sub-test-model-af081c-kserve-6fc55f5c89-kcpwb\" (UID: \"e3aea040-d014-4057-93fd-f57019a37dce\") " pod="ai-tenant-e2e-shared-a-w3-af081c/sub-test-model-af081c-kserve-6fc55f5c89-kcpwb" Sep 11 12:11:27.963117 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:27.963032 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-af081c/sub-test-model-af081c-kserve-6fc55f5c89-kcpwb" Sep 11 12:11:28.240114 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:28.240087 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-af081c/sub-test-model-af081c-kserve-6fc55f5c89-kcpwb"] Sep 11 12:11:28.241659 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:11:28.241630 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode3aea040_d014_4057_93fd_f57019a37dce.slice/crio-fab832058a4a84f6fb23493c58afe19f8c9a507a5c26b98e50904959f4113e03 WatchSource:0}: Error finding container fab832058a4a84f6fb23493c58afe19f8c9a507a5c26b98e50904959f4113e03: Status 404 returned error can't find the container with id fab832058a4a84f6fb23493c58afe19f8c9a507a5c26b98e50904959f4113e03 Sep 11 12:11:29.025428 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:29.025393 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-af081c/sub-test-model-af081c-kserve-6fc55f5c89-kcpwb" event={"ID":"e3aea040-d014-4057-93fd-f57019a37dce","Type":"ContainerStarted","Data":"fd65532c19db4ba7fbc79dbc6abca507c707c533170655ffac62d0eb18369c4a"} Sep 11 12:11:29.025428 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:29.025435 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-af081c/sub-test-model-af081c-kserve-6fc55f5c89-kcpwb" event={"ID":"e3aea040-d014-4057-93fd-f57019a37dce","Type":"ContainerStarted","Data":"fab832058a4a84f6fb23493c58afe19f8c9a507a5c26b98e50904959f4113e03"} Sep 11 12:11:29.052533 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:29.052474 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-af081c/sub-test-model-af081c-kserve-6fc55f5c89-kcpwb" podStartSLOduration=2.052454297 podStartE2EDuration="2.052454297s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:11:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:11:29.050491878 +0000 UTC m=+1420.815465056" watchObservedRunningTime="2026-09-11 12:11:29.052454297 +0000 UTC m=+1420.817427473" Sep 11 12:11:30.437620 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:30.437583 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl"] Sep 11 12:11:30.438055 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:30.437773 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" podUID="88f56cde-f00a-407c-a7cc-ba836c8a07e9" containerName="istio-proxy" containerID="cri-o://28b5ec63d12334eafe88d2da0aa88b20edb27bab4cb6c17508f86b43adbc56aa" gracePeriod=30 Sep 11 12:11:35.041997 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:35.041958 2566 patch_prober.go:28] interesting pod/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 11 12:11:35.042368 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:35.042017 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" podUID="88f56cde-f00a-407c-a7cc-ba836c8a07e9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 12:11:35.693772 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:35.693747 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" Sep 11 12:11:35.796666 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:35.796575 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88f56cde-f00a-407c-a7cc-ba836c8a07e9-workload-certs\") pod \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\" (UID: \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\") " Sep 11 12:11:35.796666 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:35.796637 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88f56cde-f00a-407c-a7cc-ba836c8a07e9-istio-token\") pod \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\" (UID: \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\") " Sep 11 12:11:35.796899 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:35.796669 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88f56cde-f00a-407c-a7cc-ba836c8a07e9-istio-envoy\") pod \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\" (UID: \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\") " Sep 11 12:11:35.796899 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:35.796715 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88f56cde-f00a-407c-a7cc-ba836c8a07e9-istio-podinfo\") pod \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\" (UID: \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\") " Sep 11 12:11:35.796899 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:35.796770 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88f56cde-f00a-407c-a7cc-ba836c8a07e9-credential-socket\") pod \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\" (UID: \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\") " Sep 11 12:11:35.796899 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:35.796800 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f97tg\" (UniqueName: \"kubernetes.io/projected/88f56cde-f00a-407c-a7cc-ba836c8a07e9-kube-api-access-f97tg\") pod \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\" (UID: \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\") " Sep 11 12:11:35.796899 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:35.796831 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88f56cde-f00a-407c-a7cc-ba836c8a07e9-istio-data\") pod \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\" (UID: \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\") " Sep 11 12:11:35.797174 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:35.796911 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88f56cde-f00a-407c-a7cc-ba836c8a07e9-istiod-ca-cert\") pod \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\" (UID: \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\") " Sep 11 12:11:35.797174 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:35.796933 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88f56cde-f00a-407c-a7cc-ba836c8a07e9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "88f56cde-f00a-407c-a7cc-ba836c8a07e9" (UID: "88f56cde-f00a-407c-a7cc-ba836c8a07e9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:11:35.797174 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:35.796957 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88f56cde-f00a-407c-a7cc-ba836c8a07e9-workload-socket\") pod \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\" (UID: \"88f56cde-f00a-407c-a7cc-ba836c8a07e9\") " Sep 11 12:11:35.797355 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:35.797194 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88f56cde-f00a-407c-a7cc-ba836c8a07e9-istio-data" (OuterVolumeSpecName: "istio-data") pod "88f56cde-f00a-407c-a7cc-ba836c8a07e9" (UID: "88f56cde-f00a-407c-a7cc-ba836c8a07e9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:11:35.797355 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:35.797270 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88f56cde-f00a-407c-a7cc-ba836c8a07e9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "88f56cde-f00a-407c-a7cc-ba836c8a07e9" (UID: "88f56cde-f00a-407c-a7cc-ba836c8a07e9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 12:11:35.797355 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:35.797289 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88f56cde-f00a-407c-a7cc-ba836c8a07e9-workload-certs\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:11:35.797355 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:35.797306 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88f56cde-f00a-407c-a7cc-ba836c8a07e9-istio-data\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:11:35.797563 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:35.797368 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88f56cde-f00a-407c-a7cc-ba836c8a07e9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "88f56cde-f00a-407c-a7cc-ba836c8a07e9" (UID: "88f56cde-f00a-407c-a7cc-ba836c8a07e9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:11:35.797563 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:35.797521 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88f56cde-f00a-407c-a7cc-ba836c8a07e9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "88f56cde-f00a-407c-a7cc-ba836c8a07e9" (UID: "88f56cde-f00a-407c-a7cc-ba836c8a07e9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:11:35.799031 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:35.799002 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88f56cde-f00a-407c-a7cc-ba836c8a07e9-istio-token" (OuterVolumeSpecName: "istio-token") pod "88f56cde-f00a-407c-a7cc-ba836c8a07e9" (UID: "88f56cde-f00a-407c-a7cc-ba836c8a07e9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:11:35.799152 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:35.799095 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/88f56cde-f00a-407c-a7cc-ba836c8a07e9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "88f56cde-f00a-407c-a7cc-ba836c8a07e9" (UID: "88f56cde-f00a-407c-a7cc-ba836c8a07e9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 11 12:11:35.799288 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:35.799264 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88f56cde-f00a-407c-a7cc-ba836c8a07e9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "88f56cde-f00a-407c-a7cc-ba836c8a07e9" (UID: "88f56cde-f00a-407c-a7cc-ba836c8a07e9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:11:35.799288 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:35.799266 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88f56cde-f00a-407c-a7cc-ba836c8a07e9-kube-api-access-f97tg" (OuterVolumeSpecName: "kube-api-access-f97tg") pod "88f56cde-f00a-407c-a7cc-ba836c8a07e9" (UID: "88f56cde-f00a-407c-a7cc-ba836c8a07e9"). InnerVolumeSpecName "kube-api-access-f97tg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:11:35.898023 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:35.897991 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88f56cde-f00a-407c-a7cc-ba836c8a07e9-istiod-ca-cert\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:11:35.898023 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:35.898021 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88f56cde-f00a-407c-a7cc-ba836c8a07e9-workload-socket\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:11:35.898229 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:35.898034 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88f56cde-f00a-407c-a7cc-ba836c8a07e9-istio-token\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:11:35.898229 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:35.898046 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88f56cde-f00a-407c-a7cc-ba836c8a07e9-istio-envoy\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:11:35.898229 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:35.898057 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88f56cde-f00a-407c-a7cc-ba836c8a07e9-istio-podinfo\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:11:35.898229 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:35.898067 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88f56cde-f00a-407c-a7cc-ba836c8a07e9-credential-socket\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:11:35.898229 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:35.898079 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f97tg\" (UniqueName: \"kubernetes.io/projected/88f56cde-f00a-407c-a7cc-ba836c8a07e9-kube-api-access-f97tg\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:11:36.059734 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:36.059641 2566 generic.go:356] "Generic (PLEG): container finished" podID="88f56cde-f00a-407c-a7cc-ba836c8a07e9" containerID="28b5ec63d12334eafe88d2da0aa88b20edb27bab4cb6c17508f86b43adbc56aa" exitCode=0 Sep 11 12:11:36.059734 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:36.059694 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" event={"ID":"88f56cde-f00a-407c-a7cc-ba836c8a07e9","Type":"ContainerDied","Data":"28b5ec63d12334eafe88d2da0aa88b20edb27bab4cb6c17508f86b43adbc56aa"} Sep 11 12:11:36.059734 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:36.059706 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" Sep 11 12:11:36.060249 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:36.059738 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl" event={"ID":"88f56cde-f00a-407c-a7cc-ba836c8a07e9","Type":"ContainerDied","Data":"c90f732b0579a1882a565eb5bd3af879caa6f34b539758294cfb008a82c32d82"} Sep 11 12:11:36.060249 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:36.059760 2566 scope.go:117] "RemoveContainer" containerID="28b5ec63d12334eafe88d2da0aa88b20edb27bab4cb6c17508f86b43adbc56aa" Sep 11 12:11:36.071078 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:36.070968 2566 scope.go:117] "RemoveContainer" containerID="448fc2c7864ecc94a7fade9ad632f0cb9f57955f7d1ffc6773c3b1f721368444" Sep 11 12:11:36.079788 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:36.079768 2566 scope.go:117] "RemoveContainer" containerID="28b5ec63d12334eafe88d2da0aa88b20edb27bab4cb6c17508f86b43adbc56aa" Sep 11 12:11:36.080162 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:11:36.080142 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28b5ec63d12334eafe88d2da0aa88b20edb27bab4cb6c17508f86b43adbc56aa\": container with ID starting with 28b5ec63d12334eafe88d2da0aa88b20edb27bab4cb6c17508f86b43adbc56aa not found: ID does not exist" containerID="28b5ec63d12334eafe88d2da0aa88b20edb27bab4cb6c17508f86b43adbc56aa" Sep 11 12:11:36.080231 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:36.080170 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28b5ec63d12334eafe88d2da0aa88b20edb27bab4cb6c17508f86b43adbc56aa"} err="failed to get container status \"28b5ec63d12334eafe88d2da0aa88b20edb27bab4cb6c17508f86b43adbc56aa\": rpc error: code = NotFound desc = could not find container \"28b5ec63d12334eafe88d2da0aa88b20edb27bab4cb6c17508f86b43adbc56aa\": container with ID starting with 28b5ec63d12334eafe88d2da0aa88b20edb27bab4cb6c17508f86b43adbc56aa not found: ID does not exist" Sep 11 12:11:36.080231 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:36.080196 2566 scope.go:117] "RemoveContainer" containerID="448fc2c7864ecc94a7fade9ad632f0cb9f57955f7d1ffc6773c3b1f721368444" Sep 11 12:11:36.080423 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:11:36.080405 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"448fc2c7864ecc94a7fade9ad632f0cb9f57955f7d1ffc6773c3b1f721368444\": container with ID starting with 448fc2c7864ecc94a7fade9ad632f0cb9f57955f7d1ffc6773c3b1f721368444 not found: ID does not exist" containerID="448fc2c7864ecc94a7fade9ad632f0cb9f57955f7d1ffc6773c3b1f721368444" Sep 11 12:11:36.080467 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:36.080429 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"448fc2c7864ecc94a7fade9ad632f0cb9f57955f7d1ffc6773c3b1f721368444"} err="failed to get container status \"448fc2c7864ecc94a7fade9ad632f0cb9f57955f7d1ffc6773c3b1f721368444\": rpc error: code = NotFound desc = could not find container \"448fc2c7864ecc94a7fade9ad632f0cb9f57955f7d1ffc6773c3b1f721368444\": container with ID starting with 448fc2c7864ecc94a7fade9ad632f0cb9f57955f7d1ffc6773c3b1f721368444 not found: ID does not exist" Sep 11 12:11:36.086331 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:36.086309 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl"] Sep 11 12:11:36.090896 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:36.090861 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-8d375e02-openshift-default-5c5df7598d-wn7xl"] Sep 11 12:11:36.759017 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:36.758984 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88f56cde-f00a-407c-a7cc-ba836c8a07e9" path="/var/lib/kubelet/pods/88f56cde-f00a-407c-a7cc-ba836c8a07e9/volumes" Sep 11 12:11:37.963410 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:37.963371 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-af081c/sub-test-model-af081c-kserve-6fc55f5c89-kcpwb" Sep 11 12:11:37.963835 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:37.963423 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-af081c/sub-test-model-af081c-kserve-6fc55f5c89-kcpwb" Sep 11 12:11:37.976057 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:37.976035 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-af081c/sub-test-model-af081c-kserve-6fc55f5c89-kcpwb" Sep 11 12:11:38.081502 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:11:38.081474 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-af081c/sub-test-model-af081c-kserve-6fc55f5c89-kcpwb" Sep 11 12:12:23.474291 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:23.474258 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7465-4b0df2"] Sep 11 12:12:23.474779 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:23.474596 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88f56cde-f00a-407c-a7cc-ba836c8a07e9" containerName="istio-proxy" Sep 11 12:12:23.474779 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:23.474607 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="88f56cde-f00a-407c-a7cc-ba836c8a07e9" containerName="istio-proxy" Sep 11 12:12:23.474779 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:23.474670 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="88f56cde-f00a-407c-a7cc-ba836c8a07e9" containerName="istio-proxy" Sep 11 12:12:23.483310 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:23.483273 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-4b0df2" Sep 11 12:12:23.485129 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:23.485105 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7465-4b0df2"] Sep 11 12:12:23.486237 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:23.486203 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-wkgmv\"" Sep 11 12:12:23.535820 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:23.535795 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgcgs\" (UniqueName: \"kubernetes.io/projected/b5ac5c18-1768-445a-a6c4-491158ec44e0-kube-api-access-qgcgs\") pod \"test-curl-7465-4b0df2\" (UID: \"b5ac5c18-1768-445a-a6c4-491158ec44e0\") " pod="opendatahub/test-curl-7465-4b0df2" Sep 11 12:12:23.637037 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:23.637009 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgcgs\" (UniqueName: \"kubernetes.io/projected/b5ac5c18-1768-445a-a6c4-491158ec44e0-kube-api-access-qgcgs\") pod \"test-curl-7465-4b0df2\" (UID: \"b5ac5c18-1768-445a-a6c4-491158ec44e0\") " pod="opendatahub/test-curl-7465-4b0df2" Sep 11 12:12:23.646255 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:23.646225 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgcgs\" (UniqueName: \"kubernetes.io/projected/b5ac5c18-1768-445a-a6c4-491158ec44e0-kube-api-access-qgcgs\") pod \"test-curl-7465-4b0df2\" (UID: \"b5ac5c18-1768-445a-a6c4-491158ec44e0\") " pod="opendatahub/test-curl-7465-4b0df2" Sep 11 12:12:23.793993 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:23.793912 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-4b0df2" Sep 11 12:12:24.067907 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:24.067853 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7465-4b0df2"] Sep 11 12:12:24.073379 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:12:24.073349 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb5ac5c18_1768_445a_a6c4_491158ec44e0.slice/crio-158500abd9a0625472bcb8b89d8ffdbe0fe75a20183de7ca88fd551ce803491b WatchSource:0}: Error finding container 158500abd9a0625472bcb8b89d8ffdbe0fe75a20183de7ca88fd551ce803491b: Status 404 returned error can't find the container with id 158500abd9a0625472bcb8b89d8ffdbe0fe75a20183de7ca88fd551ce803491b Sep 11 12:12:24.287371 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:24.287334 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7465-4b0df2" event={"ID":"b5ac5c18-1768-445a-a6c4-491158ec44e0","Type":"ContainerStarted","Data":"e73b3868ee55b4d2e2809e67b41f5eb2efde93f909d505261ee2fe1ac02876a7"} Sep 11 12:12:24.287371 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:24.287376 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7465-4b0df2" event={"ID":"b5ac5c18-1768-445a-a6c4-491158ec44e0","Type":"ContainerStarted","Data":"158500abd9a0625472bcb8b89d8ffdbe0fe75a20183de7ca88fd551ce803491b"} Sep 11 12:12:24.305773 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:24.305725 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7465-4b0df2" podStartSLOduration=1.305709188 podStartE2EDuration="1.305709188s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:12:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:12:24.303609071 +0000 UTC m=+1476.068582247" watchObservedRunningTime="2026-09-11 12:12:24.305709188 +0000 UTC m=+1476.070682363" Sep 11 12:12:24.888426 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:24.888392 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7465-4b0df2"] Sep 11 12:12:24.892920 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:24.892890 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7465-4b0df2"] Sep 11 12:12:25.292360 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:12:25.292286 2566 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7465-4b0df2" container="test-curl-7465-4b0df2" Sep 11 12:12:26.296723 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:26.296684 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7465-4b0df2" podUID="b5ac5c18-1768-445a-a6c4-491158ec44e0" containerName="test-curl-7465-4b0df2" containerID="cri-o://e73b3868ee55b4d2e2809e67b41f5eb2efde93f909d505261ee2fe1ac02876a7" gracePeriod=2 Sep 11 12:12:26.299288 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:12:26.299260 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7465-4b0df2\" is forbidden: User \"system:node:ip-10-0-132-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-170.ec2.internal' and this object" podUID="b5ac5c18-1768-445a-a6c4-491158ec44e0" pod="opendatahub/test-curl-7465-4b0df2" Sep 11 12:12:28.492990 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:28.492958 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-4b0df2" Sep 11 12:12:28.581968 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:28.581940 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qgcgs\" (UniqueName: \"kubernetes.io/projected/b5ac5c18-1768-445a-a6c4-491158ec44e0-kube-api-access-qgcgs\") pod \"b5ac5c18-1768-445a-a6c4-491158ec44e0\" (UID: \"b5ac5c18-1768-445a-a6c4-491158ec44e0\") " Sep 11 12:12:28.583932 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:28.583903 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5ac5c18-1768-445a-a6c4-491158ec44e0-kube-api-access-qgcgs" (OuterVolumeSpecName: "kube-api-access-qgcgs") pod "b5ac5c18-1768-445a-a6c4-491158ec44e0" (UID: "b5ac5c18-1768-445a-a6c4-491158ec44e0"). InnerVolumeSpecName "kube-api-access-qgcgs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:12:28.683191 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:28.683150 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgcgs\" (UniqueName: \"kubernetes.io/projected/b5ac5c18-1768-445a-a6c4-491158ec44e0-kube-api-access-qgcgs\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:12:28.759349 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:28.759315 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5ac5c18-1768-445a-a6c4-491158ec44e0" path="/var/lib/kubelet/pods/b5ac5c18-1768-445a-a6c4-491158ec44e0/volumes" Sep 11 12:12:29.217893 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:29.217833 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-af081c/sub-test-model-af081c-kserve-6fc55f5c89-kcpwb"] Sep 11 12:12:29.316716 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:29.316635 2566 generic.go:356] "Generic (PLEG): container finished" podID="b5ac5c18-1768-445a-a6c4-491158ec44e0" containerID="e73b3868ee55b4d2e2809e67b41f5eb2efde93f909d505261ee2fe1ac02876a7" exitCode=137 Sep 11 12:12:29.316987 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:29.316772 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-4b0df2" Sep 11 12:12:29.316987 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:29.316820 2566 scope.go:117] "RemoveContainer" containerID="e73b3868ee55b4d2e2809e67b41f5eb2efde93f909d505261ee2fe1ac02876a7" Sep 11 12:12:29.328849 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:29.328827 2566 scope.go:117] "RemoveContainer" containerID="e73b3868ee55b4d2e2809e67b41f5eb2efde93f909d505261ee2fe1ac02876a7" Sep 11 12:12:29.329172 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:12:29.329151 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e73b3868ee55b4d2e2809e67b41f5eb2efde93f909d505261ee2fe1ac02876a7\": container with ID starting with e73b3868ee55b4d2e2809e67b41f5eb2efde93f909d505261ee2fe1ac02876a7 not found: ID does not exist" containerID="e73b3868ee55b4d2e2809e67b41f5eb2efde93f909d505261ee2fe1ac02876a7" Sep 11 12:12:29.329240 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:29.329181 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e73b3868ee55b4d2e2809e67b41f5eb2efde93f909d505261ee2fe1ac02876a7"} err="failed to get container status \"e73b3868ee55b4d2e2809e67b41f5eb2efde93f909d505261ee2fe1ac02876a7\": rpc error: code = NotFound desc = could not find container \"e73b3868ee55b4d2e2809e67b41f5eb2efde93f909d505261ee2fe1ac02876a7\": container with ID starting with e73b3868ee55b4d2e2809e67b41f5eb2efde93f909d505261ee2fe1ac02876a7 not found: ID does not exist" Sep 11 12:12:32.268579 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:32.268540 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-fb3c56-7866c89c49-4jckp"] Sep 11 12:12:32.269111 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:32.269005 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5ac5c18-1768-445a-a6c4-491158ec44e0" containerName="test-curl-7465-4b0df2" Sep 11 12:12:32.269111 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:32.269023 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5ac5c18-1768-445a-a6c4-491158ec44e0" containerName="test-curl-7465-4b0df2" Sep 11 12:12:32.269111 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:32.269041 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88f56cde-f00a-407c-a7cc-ba836c8a07e9" containerName="istio-proxy" Sep 11 12:12:32.269111 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:32.269055 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="88f56cde-f00a-407c-a7cc-ba836c8a07e9" containerName="istio-proxy" Sep 11 12:12:32.269310 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:32.269165 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5ac5c18-1768-445a-a6c4-491158ec44e0" containerName="test-curl-7465-4b0df2" Sep 11 12:12:32.269310 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:32.269181 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="88f56cde-f00a-407c-a7cc-ba836c8a07e9" containerName="istio-proxy" Sep 11 12:12:32.273612 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:32.273592 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-fb3c56-7866c89c49-4jckp" Sep 11 12:12:32.277543 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:32.277521 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-fb3c56-dockercfg-6qfzk\"" Sep 11 12:12:32.278355 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:32.278336 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-fb3c56\"" Sep 11 12:12:32.286495 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:32.286471 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-fb3c56-7866c89c49-4jckp"] Sep 11 12:12:32.316424 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:32.316383 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8q6fq\" (UniqueName: \"kubernetes.io/projected/527b97c4-1119-403b-9774-ddcc4c116d3f-kube-api-access-8q6fq\") pod \"payload-processing-e2e-api-a-fb3c56-7866c89c49-4jckp\" (UID: \"527b97c4-1119-403b-9774-ddcc4c116d3f\") " pod="openshift-ingress/payload-processing-e2e-api-a-fb3c56-7866c89c49-4jckp" Sep 11 12:12:32.316609 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:32.316470 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/527b97c4-1119-403b-9774-ddcc4c116d3f-plugins-config-volume\") pod \"payload-processing-e2e-api-a-fb3c56-7866c89c49-4jckp\" (UID: \"527b97c4-1119-403b-9774-ddcc4c116d3f\") " pod="openshift-ingress/payload-processing-e2e-api-a-fb3c56-7866c89c49-4jckp" Sep 11 12:12:32.417740 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:32.417695 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8q6fq\" (UniqueName: \"kubernetes.io/projected/527b97c4-1119-403b-9774-ddcc4c116d3f-kube-api-access-8q6fq\") pod \"payload-processing-e2e-api-a-fb3c56-7866c89c49-4jckp\" (UID: \"527b97c4-1119-403b-9774-ddcc4c116d3f\") " pod="openshift-ingress/payload-processing-e2e-api-a-fb3c56-7866c89c49-4jckp" Sep 11 12:12:32.417977 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:32.417768 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/527b97c4-1119-403b-9774-ddcc4c116d3f-plugins-config-volume\") pod \"payload-processing-e2e-api-a-fb3c56-7866c89c49-4jckp\" (UID: \"527b97c4-1119-403b-9774-ddcc4c116d3f\") " pod="openshift-ingress/payload-processing-e2e-api-a-fb3c56-7866c89c49-4jckp" Sep 11 12:12:32.418497 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:32.418476 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/527b97c4-1119-403b-9774-ddcc4c116d3f-plugins-config-volume\") pod \"payload-processing-e2e-api-a-fb3c56-7866c89c49-4jckp\" (UID: \"527b97c4-1119-403b-9774-ddcc4c116d3f\") " pod="openshift-ingress/payload-processing-e2e-api-a-fb3c56-7866c89c49-4jckp" Sep 11 12:12:32.428313 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:32.428286 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8q6fq\" (UniqueName: \"kubernetes.io/projected/527b97c4-1119-403b-9774-ddcc4c116d3f-kube-api-access-8q6fq\") pod \"payload-processing-e2e-api-a-fb3c56-7866c89c49-4jckp\" (UID: \"527b97c4-1119-403b-9774-ddcc4c116d3f\") " pod="openshift-ingress/payload-processing-e2e-api-a-fb3c56-7866c89c49-4jckp" Sep 11 12:12:32.583814 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:32.583780 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-fb3c56-7866c89c49-4jckp" Sep 11 12:12:32.858716 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:32.858577 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-fb3c56-7866c89c49-4jckp"] Sep 11 12:12:32.861425 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:12:32.861397 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod527b97c4_1119_403b_9774_ddcc4c116d3f.slice/crio-8c59322a215ca0eb5ca0cd1cd8e980df1818f4f8154bfcacc1a2ff9beecfba4f WatchSource:0}: Error finding container 8c59322a215ca0eb5ca0cd1cd8e980df1818f4f8154bfcacc1a2ff9beecfba4f: Status 404 returned error can't find the container with id 8c59322a215ca0eb5ca0cd1cd8e980df1818f4f8154bfcacc1a2ff9beecfba4f Sep 11 12:12:33.341569 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:33.341527 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-fb3c56-7866c89c49-4jckp" event={"ID":"527b97c4-1119-403b-9774-ddcc4c116d3f","Type":"ContainerStarted","Data":"de226b83ac580bec0dee57af1c5e16b5c9f62d0884339bbf2064690b88051de8"} Sep 11 12:12:33.342086 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:33.341591 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-fb3c56-7866c89c49-4jckp" event={"ID":"527b97c4-1119-403b-9774-ddcc4c116d3f","Type":"ContainerStarted","Data":"8c59322a215ca0eb5ca0cd1cd8e980df1818f4f8154bfcacc1a2ff9beecfba4f"} Sep 11 12:12:33.342086 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:33.341630 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-fb3c56-7866c89c49-4jckp" Sep 11 12:12:33.359652 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:33.359591 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-fb3c56-7866c89c49-4jckp" podStartSLOduration=1.35956942 podStartE2EDuration="1.35956942s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:12:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:12:33.35936013 +0000 UTC m=+1485.124333305" watchObservedRunningTime="2026-09-11 12:12:33.35956942 +0000 UTC m=+1485.124542596" Sep 11 12:12:44.236917 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:44.236852 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-af081c/sub-test-model-af081c-kserve-6fc55f5c89-kcpwb" podUID="e3aea040-d014-4057-93fd-f57019a37dce" containerName="main" containerID="cri-o://fd65532c19db4ba7fbc79dbc6abca507c707c533170655ffac62d0eb18369c4a" gracePeriod=45 Sep 11 12:12:44.348062 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:44.348031 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-fb3c56-7866c89c49-4jckp" Sep 11 12:12:44.402420 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:44.402385 2566 generic.go:356] "Generic (PLEG): container finished" podID="e3aea040-d014-4057-93fd-f57019a37dce" containerID="fd65532c19db4ba7fbc79dbc6abca507c707c533170655ffac62d0eb18369c4a" exitCode=0 Sep 11 12:12:44.402609 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:44.402468 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-af081c/sub-test-model-af081c-kserve-6fc55f5c89-kcpwb" event={"ID":"e3aea040-d014-4057-93fd-f57019a37dce","Type":"ContainerDied","Data":"fd65532c19db4ba7fbc79dbc6abca507c707c533170655ffac62d0eb18369c4a"} Sep 11 12:12:44.500259 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:44.500186 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-af081c/sub-test-model-af081c-kserve-6fc55f5c89-kcpwb" Sep 11 12:12:44.635698 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:44.635660 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3aea040-d014-4057-93fd-f57019a37dce-tmp-dir\") pod \"e3aea040-d014-4057-93fd-f57019a37dce\" (UID: \"e3aea040-d014-4057-93fd-f57019a37dce\") " Sep 11 12:12:44.635698 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:44.635699 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3aea040-d014-4057-93fd-f57019a37dce-home\") pod \"e3aea040-d014-4057-93fd-f57019a37dce\" (UID: \"e3aea040-d014-4057-93fd-f57019a37dce\") " Sep 11 12:12:44.635974 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:44.635721 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3aea040-d014-4057-93fd-f57019a37dce-tls-certs\") pod \"e3aea040-d014-4057-93fd-f57019a37dce\" (UID: \"e3aea040-d014-4057-93fd-f57019a37dce\") " Sep 11 12:12:44.635974 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:44.635747 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fh2vp\" (UniqueName: \"kubernetes.io/projected/e3aea040-d014-4057-93fd-f57019a37dce-kube-api-access-fh2vp\") pod \"e3aea040-d014-4057-93fd-f57019a37dce\" (UID: \"e3aea040-d014-4057-93fd-f57019a37dce\") " Sep 11 12:12:44.635974 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:44.635767 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3aea040-d014-4057-93fd-f57019a37dce-model-cache\") pod \"e3aea040-d014-4057-93fd-f57019a37dce\" (UID: \"e3aea040-d014-4057-93fd-f57019a37dce\") " Sep 11 12:12:44.635974 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:44.635858 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3aea040-d014-4057-93fd-f57019a37dce-dshm\") pod \"e3aea040-d014-4057-93fd-f57019a37dce\" (UID: \"e3aea040-d014-4057-93fd-f57019a37dce\") " Sep 11 12:12:44.636172 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:44.636049 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3aea040-d014-4057-93fd-f57019a37dce-home" (OuterVolumeSpecName: "home") pod "e3aea040-d014-4057-93fd-f57019a37dce" (UID: "e3aea040-d014-4057-93fd-f57019a37dce"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:12:44.636172 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:44.636059 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3aea040-d014-4057-93fd-f57019a37dce-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e3aea040-d014-4057-93fd-f57019a37dce" (UID: "e3aea040-d014-4057-93fd-f57019a37dce"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:12:44.636172 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:44.636086 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3aea040-d014-4057-93fd-f57019a37dce-model-cache" (OuterVolumeSpecName: "model-cache") pod "e3aea040-d014-4057-93fd-f57019a37dce" (UID: "e3aea040-d014-4057-93fd-f57019a37dce"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:12:44.636341 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:44.636250 2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3aea040-d014-4057-93fd-f57019a37dce-tmp-dir\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:12:44.636341 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:44.636266 2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3aea040-d014-4057-93fd-f57019a37dce-home\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:12:44.636341 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:44.636274 2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3aea040-d014-4057-93fd-f57019a37dce-model-cache\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:12:44.638159 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:44.638114 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3aea040-d014-4057-93fd-f57019a37dce-dshm" (OuterVolumeSpecName: "dshm") pod "e3aea040-d014-4057-93fd-f57019a37dce" (UID: "e3aea040-d014-4057-93fd-f57019a37dce"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:12:44.638297 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:44.638181 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3aea040-d014-4057-93fd-f57019a37dce-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e3aea040-d014-4057-93fd-f57019a37dce" (UID: "e3aea040-d014-4057-93fd-f57019a37dce"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 12:12:44.638297 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:44.638192 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3aea040-d014-4057-93fd-f57019a37dce-kube-api-access-fh2vp" (OuterVolumeSpecName: "kube-api-access-fh2vp") pod "e3aea040-d014-4057-93fd-f57019a37dce" (UID: "e3aea040-d014-4057-93fd-f57019a37dce"). InnerVolumeSpecName "kube-api-access-fh2vp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:12:44.736777 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:44.736739 2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3aea040-d014-4057-93fd-f57019a37dce-tls-certs\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:12:44.736777 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:44.736773 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fh2vp\" (UniqueName: \"kubernetes.io/projected/e3aea040-d014-4057-93fd-f57019a37dce-kube-api-access-fh2vp\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:12:44.736998 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:44.736787 2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3aea040-d014-4057-93fd-f57019a37dce-dshm\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:12:45.409935 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:45.409907 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-af081c/sub-test-model-af081c-kserve-6fc55f5c89-kcpwb" Sep 11 12:12:45.409935 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:45.409908 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-af081c/sub-test-model-af081c-kserve-6fc55f5c89-kcpwb" event={"ID":"e3aea040-d014-4057-93fd-f57019a37dce","Type":"ContainerDied","Data":"fab832058a4a84f6fb23493c58afe19f8c9a507a5c26b98e50904959f4113e03"} Sep 11 12:12:45.410444 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:45.409966 2566 scope.go:117] "RemoveContainer" containerID="fd65532c19db4ba7fbc79dbc6abca507c707c533170655ffac62d0eb18369c4a" Sep 11 12:12:45.429050 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:45.429022 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-af081c/sub-test-model-af081c-kserve-6fc55f5c89-kcpwb"] Sep 11 12:12:45.434535 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:45.434513 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-af081c/sub-test-model-af081c-kserve-6fc55f5c89-kcpwb"] Sep 11 12:12:46.758979 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:46.758947 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3aea040-d014-4057-93fd-f57019a37dce" path="/var/lib/kubelet/pods/e3aea040-d014-4057-93fd-f57019a37dce/volumes" Sep 11 12:12:55.829081 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:55.829046 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx"] Sep 11 12:12:55.829475 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:55.829419 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3aea040-d014-4057-93fd-f57019a37dce" containerName="main" Sep 11 12:12:55.829475 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:55.829432 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3aea040-d014-4057-93fd-f57019a37dce" containerName="main" Sep 11 12:12:55.829568 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:55.829505 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3aea040-d014-4057-93fd-f57019a37dce" containerName="main" Sep 11 12:12:55.832968 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:55.832949 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" Sep 11 12:12:55.837315 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:55.837293 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-f4d189-openshift-default-dockercfg-zfd9z\"" Sep 11 12:12:55.846179 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:55.846155 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx"] Sep 11 12:12:55.937326 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:55.937287 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99474cac-4e87-491e-9943-639595a9b084-istiod-ca-cert\") pod \"e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx\" (UID: \"99474cac-4e87-491e-9943-639595a9b084\") " pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" Sep 11 12:12:55.937499 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:55.937338 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99474cac-4e87-491e-9943-639595a9b084-istio-podinfo\") pod \"e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx\" (UID: \"99474cac-4e87-491e-9943-639595a9b084\") " pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" Sep 11 12:12:55.937499 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:55.937406 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99474cac-4e87-491e-9943-639595a9b084-istio-envoy\") pod \"e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx\" (UID: \"99474cac-4e87-491e-9943-639595a9b084\") " pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" Sep 11 12:12:55.937499 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:55.937435 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99474cac-4e87-491e-9943-639595a9b084-istio-token\") pod \"e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx\" (UID: \"99474cac-4e87-491e-9943-639595a9b084\") " pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" Sep 11 12:12:55.937499 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:55.937473 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99474cac-4e87-491e-9943-639595a9b084-workload-socket\") pod \"e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx\" (UID: \"99474cac-4e87-491e-9943-639595a9b084\") " pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" Sep 11 12:12:55.937678 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:55.937511 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99474cac-4e87-491e-9943-639595a9b084-istio-data\") pod \"e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx\" (UID: \"99474cac-4e87-491e-9943-639595a9b084\") " pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" Sep 11 12:12:55.937678 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:55.937547 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99474cac-4e87-491e-9943-639595a9b084-credential-socket\") pod \"e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx\" (UID: \"99474cac-4e87-491e-9943-639595a9b084\") " pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" Sep 11 12:12:55.937678 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:55.937590 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnxwr\" (UniqueName: \"kubernetes.io/projected/99474cac-4e87-491e-9943-639595a9b084-kube-api-access-bnxwr\") pod \"e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx\" (UID: \"99474cac-4e87-491e-9943-639595a9b084\") " pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" Sep 11 12:12:55.937678 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:55.937615 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99474cac-4e87-491e-9943-639595a9b084-workload-certs\") pod \"e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx\" (UID: \"99474cac-4e87-491e-9943-639595a9b084\") " pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" Sep 11 12:12:56.038445 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:56.038407 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99474cac-4e87-491e-9943-639595a9b084-istio-envoy\") pod \"e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx\" (UID: \"99474cac-4e87-491e-9943-639595a9b084\") " pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" Sep 11 12:12:56.038445 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:56.038443 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99474cac-4e87-491e-9943-639595a9b084-istio-token\") pod \"e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx\" (UID: \"99474cac-4e87-491e-9943-639595a9b084\") " pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" Sep 11 12:12:56.038672 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:56.038470 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99474cac-4e87-491e-9943-639595a9b084-workload-socket\") pod \"e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx\" (UID: \"99474cac-4e87-491e-9943-639595a9b084\") " pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" Sep 11 12:12:56.038672 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:56.038502 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99474cac-4e87-491e-9943-639595a9b084-istio-data\") pod \"e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx\" (UID: \"99474cac-4e87-491e-9943-639595a9b084\") " pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" Sep 11 12:12:56.038672 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:56.038535 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99474cac-4e87-491e-9943-639595a9b084-credential-socket\") pod \"e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx\" (UID: \"99474cac-4e87-491e-9943-639595a9b084\") " pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" Sep 11 12:12:56.038672 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:56.038575 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnxwr\" (UniqueName: \"kubernetes.io/projected/99474cac-4e87-491e-9943-639595a9b084-kube-api-access-bnxwr\") pod \"e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx\" (UID: \"99474cac-4e87-491e-9943-639595a9b084\") " pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" Sep 11 12:12:56.038672 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:56.038604 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99474cac-4e87-491e-9943-639595a9b084-workload-certs\") pod \"e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx\" (UID: \"99474cac-4e87-491e-9943-639595a9b084\") " pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" Sep 11 12:12:56.038979 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:56.038688 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99474cac-4e87-491e-9943-639595a9b084-istiod-ca-cert\") pod \"e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx\" (UID: \"99474cac-4e87-491e-9943-639595a9b084\") " pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" Sep 11 12:12:56.038979 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:56.038725 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99474cac-4e87-491e-9943-639595a9b084-istio-podinfo\") pod \"e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx\" (UID: \"99474cac-4e87-491e-9943-639595a9b084\") " pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" Sep 11 12:12:56.038979 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:56.038948 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99474cac-4e87-491e-9943-639595a9b084-workload-socket\") pod \"e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx\" (UID: \"99474cac-4e87-491e-9943-639595a9b084\") " pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" Sep 11 12:12:56.039120 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:56.039094 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99474cac-4e87-491e-9943-639595a9b084-istio-data\") pod \"e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx\" (UID: \"99474cac-4e87-491e-9943-639595a9b084\") " pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" Sep 11 12:12:56.039159 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:56.039094 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99474cac-4e87-491e-9943-639595a9b084-credential-socket\") pod \"e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx\" (UID: \"99474cac-4e87-491e-9943-639595a9b084\") " pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" Sep 11 12:12:56.039159 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:56.039137 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99474cac-4e87-491e-9943-639595a9b084-workload-certs\") pod \"e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx\" (UID: \"99474cac-4e87-491e-9943-639595a9b084\") " pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" Sep 11 12:12:56.039612 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:56.039585 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99474cac-4e87-491e-9943-639595a9b084-istiod-ca-cert\") pod \"e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx\" (UID: \"99474cac-4e87-491e-9943-639595a9b084\") " pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" Sep 11 12:12:56.040952 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:56.040931 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99474cac-4e87-491e-9943-639595a9b084-istio-envoy\") pod \"e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx\" (UID: \"99474cac-4e87-491e-9943-639595a9b084\") " pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" Sep 11 12:12:56.041101 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:56.041086 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99474cac-4e87-491e-9943-639595a9b084-istio-podinfo\") pod \"e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx\" (UID: \"99474cac-4e87-491e-9943-639595a9b084\") " pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" Sep 11 12:12:56.048298 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:56.048273 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnxwr\" (UniqueName: \"kubernetes.io/projected/99474cac-4e87-491e-9943-639595a9b084-kube-api-access-bnxwr\") pod \"e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx\" (UID: \"99474cac-4e87-491e-9943-639595a9b084\") " pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" Sep 11 12:12:56.048298 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:56.048292 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99474cac-4e87-491e-9943-639595a9b084-istio-token\") pod \"e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx\" (UID: \"99474cac-4e87-491e-9943-639595a9b084\") " pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" Sep 11 12:12:56.142971 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:56.142889 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" Sep 11 12:12:56.423119 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:56.423091 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx"] Sep 11 12:12:56.426330 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:12:56.426297 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99474cac_4e87_491e_9943_639595a9b084.slice/crio-4a6f8749e04823aef451324a7c4adf686cbb635f310ccfd56922409b103ba011 WatchSource:0}: Error finding container 4a6f8749e04823aef451324a7c4adf686cbb635f310ccfd56922409b103ba011: Status 404 returned error can't find the container with id 4a6f8749e04823aef451324a7c4adf686cbb635f310ccfd56922409b103ba011 Sep 11 12:12:56.429187 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:56.429152 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:12:56.429260 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:56.429222 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:12:56.429260 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:56.429253 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:12:56.464966 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:56.464938 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" event={"ID":"99474cac-4e87-491e-9943-639595a9b084","Type":"ContainerStarted","Data":"4a6f8749e04823aef451324a7c4adf686cbb635f310ccfd56922409b103ba011"} Sep 11 12:12:57.471579 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:57.471540 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" event={"ID":"99474cac-4e87-491e-9943-639595a9b084","Type":"ContainerStarted","Data":"6b5d0a16f76c1d897a050c394ef7c0cf9f3b8e2e281d78d6ab44825f94d4a531"} Sep 11 12:12:57.494250 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:57.494201 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" podStartSLOduration=2.494185962 podStartE2EDuration="2.494185962s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:12:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:12:57.492267783 +0000 UTC m=+1509.257240960" watchObservedRunningTime="2026-09-11 12:12:57.494185962 +0000 UTC m=+1509.259159137" Sep 11 12:12:58.144040 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:58.144003 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" Sep 11 12:12:58.149224 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:58.149197 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" Sep 11 12:12:58.482424 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:58.482336 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" Sep 11 12:12:58.483301 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:12:58.483282 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" Sep 11 12:13:01.327975 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:01.327940 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f4d189-674956cf6b-lkpgd"] Sep 11 12:13:01.331130 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:01.331111 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f4d189-674956cf6b-lkpgd" Sep 11 12:13:01.333746 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:01.333710 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-f4d189\"" Sep 11 12:13:01.333903 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:01.333799 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-f4d189-dockercfg-cv2cq\"" Sep 11 12:13:01.344310 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:01.344282 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f4d189-674956cf6b-lkpgd"] Sep 11 12:13:01.387394 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:01.387357 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9s9r\" (UniqueName: \"kubernetes.io/projected/f6f493b3-3194-4a25-8dba-71b9820cba82-kube-api-access-g9s9r\") pod \"payload-pre-processing-e2e-api-b-f4d189-674956cf6b-lkpgd\" (UID: \"f6f493b3-3194-4a25-8dba-71b9820cba82\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f4d189-674956cf6b-lkpgd" Sep 11 12:13:01.387579 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:01.387412 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6f493b3-3194-4a25-8dba-71b9820cba82-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-f4d189-674956cf6b-lkpgd\" (UID: \"f6f493b3-3194-4a25-8dba-71b9820cba82\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f4d189-674956cf6b-lkpgd" Sep 11 12:13:01.488589 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:01.488550 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6f493b3-3194-4a25-8dba-71b9820cba82-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-f4d189-674956cf6b-lkpgd\" (UID: \"f6f493b3-3194-4a25-8dba-71b9820cba82\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f4d189-674956cf6b-lkpgd" Sep 11 12:13:01.488798 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:01.488703 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9s9r\" (UniqueName: \"kubernetes.io/projected/f6f493b3-3194-4a25-8dba-71b9820cba82-kube-api-access-g9s9r\") pod \"payload-pre-processing-e2e-api-b-f4d189-674956cf6b-lkpgd\" (UID: \"f6f493b3-3194-4a25-8dba-71b9820cba82\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f4d189-674956cf6b-lkpgd" Sep 11 12:13:01.489294 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:01.489270 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6f493b3-3194-4a25-8dba-71b9820cba82-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-f4d189-674956cf6b-lkpgd\" (UID: \"f6f493b3-3194-4a25-8dba-71b9820cba82\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f4d189-674956cf6b-lkpgd" Sep 11 12:13:01.497095 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:01.497062 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9s9r\" (UniqueName: \"kubernetes.io/projected/f6f493b3-3194-4a25-8dba-71b9820cba82-kube-api-access-g9s9r\") pod \"payload-pre-processing-e2e-api-b-f4d189-674956cf6b-lkpgd\" (UID: \"f6f493b3-3194-4a25-8dba-71b9820cba82\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f4d189-674956cf6b-lkpgd" Sep 11 12:13:01.641936 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:01.641825 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f4d189-674956cf6b-lkpgd" Sep 11 12:13:01.918145 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:01.918118 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f4d189-674956cf6b-lkpgd"] Sep 11 12:13:01.920611 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:13:01.920575 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6f493b3_3194_4a25_8dba_71b9820cba82.slice/crio-7099a5a69cafe513c3d3f2bd399eee037bb1f6c2750184a956ba38c0d7b90a23 WatchSource:0}: Error finding container 7099a5a69cafe513c3d3f2bd399eee037bb1f6c2750184a956ba38c0d7b90a23: Status 404 returned error can't find the container with id 7099a5a69cafe513c3d3f2bd399eee037bb1f6c2750184a956ba38c0d7b90a23 Sep 11 12:13:02.503083 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:02.503045 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f4d189-674956cf6b-lkpgd" event={"ID":"f6f493b3-3194-4a25-8dba-71b9820cba82","Type":"ContainerStarted","Data":"e12edcadb5bd1a2386e72d47ebc8ab399213589df132a308ac20d67763123a42"} Sep 11 12:13:02.503526 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:02.503113 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f4d189-674956cf6b-lkpgd" Sep 11 12:13:02.503526 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:02.503142 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f4d189-674956cf6b-lkpgd" event={"ID":"f6f493b3-3194-4a25-8dba-71b9820cba82","Type":"ContainerStarted","Data":"7099a5a69cafe513c3d3f2bd399eee037bb1f6c2750184a956ba38c0d7b90a23"} Sep 11 12:13:02.523789 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:02.523729 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f4d189-674956cf6b-lkpgd" podStartSLOduration=1.523708836 podStartE2EDuration="1.523708836s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:13:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:13:02.523524494 +0000 UTC m=+1514.288497682" watchObservedRunningTime="2026-09-11 12:13:02.523708836 +0000 UTC m=+1514.288682013" Sep 11 12:13:13.511614 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:13.511582 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f4d189-674956cf6b-lkpgd" Sep 11 12:13:14.523034 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:14.522991 2566 patch_prober.go:28] interesting pod/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 11 12:13:14.523499 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:14.523068 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" podUID="99474cac-4e87-491e-9943-639595a9b084" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 11 12:13:15.572809 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:15.572775 2566 generic.go:356] "Generic (PLEG): container finished" podID="99474cac-4e87-491e-9943-639595a9b084" containerID="6b5d0a16f76c1d897a050c394ef7c0cf9f3b8e2e281d78d6ab44825f94d4a531" exitCode=0 Sep 11 12:13:15.573234 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:15.572847 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" event={"ID":"99474cac-4e87-491e-9943-639595a9b084","Type":"ContainerDied","Data":"6b5d0a16f76c1d897a050c394ef7c0cf9f3b8e2e281d78d6ab44825f94d4a531"} Sep 11 12:13:15.573273 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:15.573254 2566 scope.go:117] "RemoveContainer" containerID="6b5d0a16f76c1d897a050c394ef7c0cf9f3b8e2e281d78d6ab44825f94d4a531" Sep 11 12:13:15.574371 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:15.574343 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:13:15.574430 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:15.574416 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:13:15.574465 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:15.574446 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:13:16.579667 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:16.579620 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" event={"ID":"99474cac-4e87-491e-9943-639595a9b084","Type":"ContainerStarted","Data":"34fb4b8dfe46e0dd55307c8f0345e52bd938883da9c099f176354c7100780ecb"} Sep 11 12:13:17.143247 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:17.143206 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" Sep 11 12:13:17.143472 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:17.143453 2566 patch_prober.go:28] interesting pod/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Sep 11 12:13:17.143530 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:17.143497 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" podUID="99474cac-4e87-491e-9943-639595a9b084" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Sep 11 12:13:18.143483 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:18.143450 2566 patch_prober.go:28] interesting pod/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Sep 11 12:13:18.143889 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:18.143505 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" podUID="99474cac-4e87-491e-9943-639595a9b084" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Sep 11 12:13:19.143474 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:19.143435 2566 patch_prober.go:28] interesting pod/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Sep 11 12:13:19.143651 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:19.143498 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" podUID="99474cac-4e87-491e-9943-639595a9b084" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Sep 11 12:13:20.143520 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:20.143478 2566 patch_prober.go:28] interesting pod/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Sep 11 12:13:20.143709 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:20.143541 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" podUID="99474cac-4e87-491e-9943-639595a9b084" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Sep 11 12:13:21.143435 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:21.143393 2566 patch_prober.go:28] interesting pod/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Sep 11 12:13:21.143615 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:21.143453 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" podUID="99474cac-4e87-491e-9943-639595a9b084" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Sep 11 12:13:22.147582 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:22.147547 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" Sep 11 12:13:22.147957 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:22.147817 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" Sep 11 12:13:22.148415 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:22.148398 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" Sep 11 12:13:42.661401 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:42.661360 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-fb3c56-7866c89c49-4jckp"] Sep 11 12:13:42.662387 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:42.662332 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-fb3c56-7866c89c49-4jckp" podUID="527b97c4-1119-403b-9774-ddcc4c116d3f" containerName="payload-processing" containerID="cri-o://de226b83ac580bec0dee57af1c5e16b5c9f62d0884339bbf2064690b88051de8" gracePeriod=30 Sep 11 12:13:42.919006 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:42.918944 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-fb3c56-7866c89c49-4jckp" Sep 11 12:13:42.973123 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:42.973096 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8q6fq\" (UniqueName: \"kubernetes.io/projected/527b97c4-1119-403b-9774-ddcc4c116d3f-kube-api-access-8q6fq\") pod \"527b97c4-1119-403b-9774-ddcc4c116d3f\" (UID: \"527b97c4-1119-403b-9774-ddcc4c116d3f\") " Sep 11 12:13:42.973288 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:42.973171 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/527b97c4-1119-403b-9774-ddcc4c116d3f-plugins-config-volume\") pod \"527b97c4-1119-403b-9774-ddcc4c116d3f\" (UID: \"527b97c4-1119-403b-9774-ddcc4c116d3f\") " Sep 11 12:13:42.973582 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:42.973549 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/527b97c4-1119-403b-9774-ddcc4c116d3f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "527b97c4-1119-403b-9774-ddcc4c116d3f" (UID: "527b97c4-1119-403b-9774-ddcc4c116d3f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 12:13:42.975425 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:42.975400 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/527b97c4-1119-403b-9774-ddcc4c116d3f-kube-api-access-8q6fq" (OuterVolumeSpecName: "kube-api-access-8q6fq") pod "527b97c4-1119-403b-9774-ddcc4c116d3f" (UID: "527b97c4-1119-403b-9774-ddcc4c116d3f"). InnerVolumeSpecName "kube-api-access-8q6fq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:13:43.074212 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:43.074171 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/527b97c4-1119-403b-9774-ddcc4c116d3f-plugins-config-volume\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:13:43.074212 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:43.074204 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8q6fq\" (UniqueName: \"kubernetes.io/projected/527b97c4-1119-403b-9774-ddcc4c116d3f-kube-api-access-8q6fq\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:13:43.725566 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:43.725533 2566 generic.go:356] "Generic (PLEG): container finished" podID="527b97c4-1119-403b-9774-ddcc4c116d3f" containerID="de226b83ac580bec0dee57af1c5e16b5c9f62d0884339bbf2064690b88051de8" exitCode=0 Sep 11 12:13:43.725986 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:43.725583 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-fb3c56-7866c89c49-4jckp" Sep 11 12:13:43.725986 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:43.725623 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-fb3c56-7866c89c49-4jckp" event={"ID":"527b97c4-1119-403b-9774-ddcc4c116d3f","Type":"ContainerDied","Data":"de226b83ac580bec0dee57af1c5e16b5c9f62d0884339bbf2064690b88051de8"} Sep 11 12:13:43.725986 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:43.725676 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-fb3c56-7866c89c49-4jckp" event={"ID":"527b97c4-1119-403b-9774-ddcc4c116d3f","Type":"ContainerDied","Data":"8c59322a215ca0eb5ca0cd1cd8e980df1818f4f8154bfcacc1a2ff9beecfba4f"} Sep 11 12:13:43.725986 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:43.725700 2566 scope.go:117] "RemoveContainer" containerID="de226b83ac580bec0dee57af1c5e16b5c9f62d0884339bbf2064690b88051de8" Sep 11 12:13:43.734700 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:43.734683 2566 scope.go:117] "RemoveContainer" containerID="de226b83ac580bec0dee57af1c5e16b5c9f62d0884339bbf2064690b88051de8" Sep 11 12:13:43.734975 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:13:43.734954 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de226b83ac580bec0dee57af1c5e16b5c9f62d0884339bbf2064690b88051de8\": container with ID starting with de226b83ac580bec0dee57af1c5e16b5c9f62d0884339bbf2064690b88051de8 not found: ID does not exist" containerID="de226b83ac580bec0dee57af1c5e16b5c9f62d0884339bbf2064690b88051de8" Sep 11 12:13:43.735038 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:43.734983 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de226b83ac580bec0dee57af1c5e16b5c9f62d0884339bbf2064690b88051de8"} err="failed to get container status \"de226b83ac580bec0dee57af1c5e16b5c9f62d0884339bbf2064690b88051de8\": rpc error: code = NotFound desc = could not find container \"de226b83ac580bec0dee57af1c5e16b5c9f62d0884339bbf2064690b88051de8\": container with ID starting with de226b83ac580bec0dee57af1c5e16b5c9f62d0884339bbf2064690b88051de8 not found: ID does not exist" Sep 11 12:13:43.748787 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:43.748764 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-fb3c56-7866c89c49-4jckp"] Sep 11 12:13:43.751777 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:43.751755 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-fb3c56-7866c89c49-4jckp"] Sep 11 12:13:44.758656 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:13:44.758619 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="527b97c4-1119-403b-9774-ddcc4c116d3f" path="/var/lib/kubelet/pods/527b97c4-1119-403b-9774-ddcc4c116d3f/volumes" Sep 11 12:14:05.827101 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:05.827067 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f4d189-674956cf6b-lkpgd"] Sep 11 12:14:05.827604 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:05.827326 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f4d189-674956cf6b-lkpgd" podUID="f6f493b3-3194-4a25-8dba-71b9820cba82" containerName="payload-pre-processing" containerID="cri-o://e12edcadb5bd1a2386e72d47ebc8ab399213589df132a308ac20d67763123a42" gracePeriod=30 Sep 11 12:14:06.082408 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:06.082344 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f4d189-674956cf6b-lkpgd" Sep 11 12:14:06.186506 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:06.186142 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6f493b3-3194-4a25-8dba-71b9820cba82-plugins-config-volume\") pod \"f6f493b3-3194-4a25-8dba-71b9820cba82\" (UID: \"f6f493b3-3194-4a25-8dba-71b9820cba82\") " Sep 11 12:14:06.186506 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:06.186190 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g9s9r\" (UniqueName: \"kubernetes.io/projected/f6f493b3-3194-4a25-8dba-71b9820cba82-kube-api-access-g9s9r\") pod \"f6f493b3-3194-4a25-8dba-71b9820cba82\" (UID: \"f6f493b3-3194-4a25-8dba-71b9820cba82\") " Sep 11 12:14:06.186732 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:06.186616 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6f493b3-3194-4a25-8dba-71b9820cba82-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f6f493b3-3194-4a25-8dba-71b9820cba82" (UID: "f6f493b3-3194-4a25-8dba-71b9820cba82"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 12:14:06.188921 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:06.188855 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6f493b3-3194-4a25-8dba-71b9820cba82-kube-api-access-g9s9r" (OuterVolumeSpecName: "kube-api-access-g9s9r") pod "f6f493b3-3194-4a25-8dba-71b9820cba82" (UID: "f6f493b3-3194-4a25-8dba-71b9820cba82"). InnerVolumeSpecName "kube-api-access-g9s9r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:14:06.287405 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:06.287368 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6f493b3-3194-4a25-8dba-71b9820cba82-plugins-config-volume\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:14:06.287405 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:06.287399 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9s9r\" (UniqueName: \"kubernetes.io/projected/f6f493b3-3194-4a25-8dba-71b9820cba82-kube-api-access-g9s9r\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:14:06.846188 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:06.846153 2566 generic.go:356] "Generic (PLEG): container finished" podID="f6f493b3-3194-4a25-8dba-71b9820cba82" containerID="e12edcadb5bd1a2386e72d47ebc8ab399213589df132a308ac20d67763123a42" exitCode=0 Sep 11 12:14:06.846602 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:06.846200 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f4d189-674956cf6b-lkpgd" Sep 11 12:14:06.846602 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:06.846204 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f4d189-674956cf6b-lkpgd" event={"ID":"f6f493b3-3194-4a25-8dba-71b9820cba82","Type":"ContainerDied","Data":"e12edcadb5bd1a2386e72d47ebc8ab399213589df132a308ac20d67763123a42"} Sep 11 12:14:06.846602 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:06.846305 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f4d189-674956cf6b-lkpgd" event={"ID":"f6f493b3-3194-4a25-8dba-71b9820cba82","Type":"ContainerDied","Data":"7099a5a69cafe513c3d3f2bd399eee037bb1f6c2750184a956ba38c0d7b90a23"} Sep 11 12:14:06.846602 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:06.846328 2566 scope.go:117] "RemoveContainer" containerID="e12edcadb5bd1a2386e72d47ebc8ab399213589df132a308ac20d67763123a42" Sep 11 12:14:06.854857 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:06.854840 2566 scope.go:117] "RemoveContainer" containerID="e12edcadb5bd1a2386e72d47ebc8ab399213589df132a308ac20d67763123a42" Sep 11 12:14:06.855165 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:14:06.855144 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e12edcadb5bd1a2386e72d47ebc8ab399213589df132a308ac20d67763123a42\": container with ID starting with e12edcadb5bd1a2386e72d47ebc8ab399213589df132a308ac20d67763123a42 not found: ID does not exist" containerID="e12edcadb5bd1a2386e72d47ebc8ab399213589df132a308ac20d67763123a42" Sep 11 12:14:06.855209 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:06.855174 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e12edcadb5bd1a2386e72d47ebc8ab399213589df132a308ac20d67763123a42"} err="failed to get container status \"e12edcadb5bd1a2386e72d47ebc8ab399213589df132a308ac20d67763123a42\": rpc error: code = NotFound desc = could not find container \"e12edcadb5bd1a2386e72d47ebc8ab399213589df132a308ac20d67763123a42\": container with ID starting with e12edcadb5bd1a2386e72d47ebc8ab399213589df132a308ac20d67763123a42 not found: ID does not exist" Sep 11 12:14:06.862349 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:06.862326 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f4d189-674956cf6b-lkpgd"] Sep 11 12:14:06.866323 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:06.866304 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f4d189-674956cf6b-lkpgd"] Sep 11 12:14:07.443367 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:07.443326 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-bf9d5c-7c46789f87-94c7j"] Sep 11 12:14:07.444137 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:07.444109 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="527b97c4-1119-403b-9774-ddcc4c116d3f" containerName="payload-processing" Sep 11 12:14:07.444137 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:07.444136 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="527b97c4-1119-403b-9774-ddcc4c116d3f" containerName="payload-processing" Sep 11 12:14:07.444336 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:07.444166 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6f493b3-3194-4a25-8dba-71b9820cba82" containerName="payload-pre-processing" Sep 11 12:14:07.444336 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:07.444175 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6f493b3-3194-4a25-8dba-71b9820cba82" containerName="payload-pre-processing" Sep 11 12:14:07.444431 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:07.444355 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="527b97c4-1119-403b-9774-ddcc4c116d3f" containerName="payload-processing" Sep 11 12:14:07.444431 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:07.444369 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6f493b3-3194-4a25-8dba-71b9820cba82" containerName="payload-pre-processing" Sep 11 12:14:07.449399 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:07.449374 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-bf9d5c-7c46789f87-94c7j" Sep 11 12:14:07.452114 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:07.452090 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-bf9d5c\"" Sep 11 12:14:07.452222 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:07.452135 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-bf9d5c-dockercfg-fm8sk\"" Sep 11 12:14:07.457223 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:07.457197 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-bf9d5c-7c46789f87-94c7j"] Sep 11 12:14:07.601557 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:07.601515 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74844184-6bfc-476f-b940-14bbb7018f24-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-bf9d5c-7c46789f87-94c7j\" (UID: \"74844184-6bfc-476f-b940-14bbb7018f24\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-bf9d5c-7c46789f87-94c7j" Sep 11 12:14:07.601730 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:07.601663 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ngfd\" (UniqueName: \"kubernetes.io/projected/74844184-6bfc-476f-b940-14bbb7018f24-kube-api-access-7ngfd\") pod \"payload-processing-e2e-ipp-a-bf9d5c-7c46789f87-94c7j\" (UID: \"74844184-6bfc-476f-b940-14bbb7018f24\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-bf9d5c-7c46789f87-94c7j" Sep 11 12:14:07.702279 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:07.702198 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ngfd\" (UniqueName: \"kubernetes.io/projected/74844184-6bfc-476f-b940-14bbb7018f24-kube-api-access-7ngfd\") pod \"payload-processing-e2e-ipp-a-bf9d5c-7c46789f87-94c7j\" (UID: \"74844184-6bfc-476f-b940-14bbb7018f24\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-bf9d5c-7c46789f87-94c7j" Sep 11 12:14:07.702425 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:07.702288 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74844184-6bfc-476f-b940-14bbb7018f24-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-bf9d5c-7c46789f87-94c7j\" (UID: \"74844184-6bfc-476f-b940-14bbb7018f24\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-bf9d5c-7c46789f87-94c7j" Sep 11 12:14:07.703040 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:07.703020 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74844184-6bfc-476f-b940-14bbb7018f24-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-bf9d5c-7c46789f87-94c7j\" (UID: \"74844184-6bfc-476f-b940-14bbb7018f24\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-bf9d5c-7c46789f87-94c7j" Sep 11 12:14:07.710999 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:07.710974 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ngfd\" (UniqueName: \"kubernetes.io/projected/74844184-6bfc-476f-b940-14bbb7018f24-kube-api-access-7ngfd\") pod \"payload-processing-e2e-ipp-a-bf9d5c-7c46789f87-94c7j\" (UID: \"74844184-6bfc-476f-b940-14bbb7018f24\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-bf9d5c-7c46789f87-94c7j" Sep 11 12:14:07.762053 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:07.762024 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-bf9d5c-7c46789f87-94c7j" Sep 11 12:14:08.034292 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:08.034264 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-bf9d5c-7c46789f87-94c7j"] Sep 11 12:14:08.036766 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:14:08.036738 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74844184_6bfc_476f_b940_14bbb7018f24.slice/crio-31efd6346aef59a74b049f22349b12a4bb1679cc34eaf61b95e9657825de53d9 WatchSource:0}: Error finding container 31efd6346aef59a74b049f22349b12a4bb1679cc34eaf61b95e9657825de53d9: Status 404 returned error can't find the container with id 31efd6346aef59a74b049f22349b12a4bb1679cc34eaf61b95e9657825de53d9 Sep 11 12:14:08.758320 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:08.758285 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6f493b3-3194-4a25-8dba-71b9820cba82" path="/var/lib/kubelet/pods/f6f493b3-3194-4a25-8dba-71b9820cba82/volumes" Sep 11 12:14:08.858630 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:08.858593 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-bf9d5c-7c46789f87-94c7j" event={"ID":"74844184-6bfc-476f-b940-14bbb7018f24","Type":"ContainerStarted","Data":"2f05796f30bb3681f9365f6bc7d6a5d7a3f8177a14b12f71e8c81896b932d7aa"} Sep 11 12:14:08.858813 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:08.858660 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-bf9d5c-7c46789f87-94c7j" Sep 11 12:14:08.858813 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:08.858683 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-bf9d5c-7c46789f87-94c7j" event={"ID":"74844184-6bfc-476f-b940-14bbb7018f24","Type":"ContainerStarted","Data":"31efd6346aef59a74b049f22349b12a4bb1679cc34eaf61b95e9657825de53d9"} Sep 11 12:14:08.878726 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:08.878679 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-bf9d5c-7c46789f87-94c7j" podStartSLOduration=1.878663624 podStartE2EDuration="1.878663624s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:14:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:14:08.877176862 +0000 UTC m=+1580.642150052" watchObservedRunningTime="2026-09-11 12:14:08.878663624 +0000 UTC m=+1580.643636798" Sep 11 12:14:12.676525 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:12.676493 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx"] Sep 11 12:14:12.676998 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:12.676759 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" podUID="99474cac-4e87-491e-9943-639595a9b084" containerName="istio-proxy" containerID="cri-o://34fb4b8dfe46e0dd55307c8f0345e52bd938883da9c099f176354c7100780ecb" gracePeriod=30 Sep 11 12:14:17.906118 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:17.906082 2566 generic.go:356] "Generic (PLEG): container finished" podID="99474cac-4e87-491e-9943-639595a9b084" containerID="34fb4b8dfe46e0dd55307c8f0345e52bd938883da9c099f176354c7100780ecb" exitCode=0 Sep 11 12:14:17.906590 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:17.906164 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" event={"ID":"99474cac-4e87-491e-9943-639595a9b084","Type":"ContainerDied","Data":"34fb4b8dfe46e0dd55307c8f0345e52bd938883da9c099f176354c7100780ecb"} Sep 11 12:14:17.906590 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:17.906239 2566 scope.go:117] "RemoveContainer" containerID="6b5d0a16f76c1d897a050c394ef7c0cf9f3b8e2e281d78d6ab44825f94d4a531" Sep 11 12:14:17.933258 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:17.933239 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" Sep 11 12:14:18.099312 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:18.099282 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99474cac-4e87-491e-9943-639595a9b084-istiod-ca-cert\") pod \"99474cac-4e87-491e-9943-639595a9b084\" (UID: \"99474cac-4e87-491e-9943-639595a9b084\") " Sep 11 12:14:18.099526 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:18.099345 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99474cac-4e87-491e-9943-639595a9b084-istio-envoy\") pod \"99474cac-4e87-491e-9943-639595a9b084\" (UID: \"99474cac-4e87-491e-9943-639595a9b084\") " Sep 11 12:14:18.099526 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:18.099395 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99474cac-4e87-491e-9943-639595a9b084-credential-socket\") pod \"99474cac-4e87-491e-9943-639595a9b084\" (UID: \"99474cac-4e87-491e-9943-639595a9b084\") " Sep 11 12:14:18.099526 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:18.099419 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99474cac-4e87-491e-9943-639595a9b084-istio-data\") pod \"99474cac-4e87-491e-9943-639595a9b084\" (UID: \"99474cac-4e87-491e-9943-639595a9b084\") " Sep 11 12:14:18.099526 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:18.099452 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bnxwr\" (UniqueName: \"kubernetes.io/projected/99474cac-4e87-491e-9943-639595a9b084-kube-api-access-bnxwr\") pod \"99474cac-4e87-491e-9943-639595a9b084\" (UID: \"99474cac-4e87-491e-9943-639595a9b084\") " Sep 11 12:14:18.099526 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:18.099488 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99474cac-4e87-491e-9943-639595a9b084-workload-socket\") pod \"99474cac-4e87-491e-9943-639595a9b084\" (UID: \"99474cac-4e87-491e-9943-639595a9b084\") " Sep 11 12:14:18.099526 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:18.099512 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99474cac-4e87-491e-9943-639595a9b084-workload-certs\") pod \"99474cac-4e87-491e-9943-639595a9b084\" (UID: \"99474cac-4e87-491e-9943-639595a9b084\") " Sep 11 12:14:18.099816 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:18.099574 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99474cac-4e87-491e-9943-639595a9b084-istio-token\") pod \"99474cac-4e87-491e-9943-639595a9b084\" (UID: \"99474cac-4e87-491e-9943-639595a9b084\") " Sep 11 12:14:18.099816 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:18.099603 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99474cac-4e87-491e-9943-639595a9b084-istio-podinfo\") pod \"99474cac-4e87-491e-9943-639595a9b084\" (UID: \"99474cac-4e87-491e-9943-639595a9b084\") " Sep 11 12:14:18.099816 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:18.099708 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99474cac-4e87-491e-9943-639595a9b084-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "99474cac-4e87-491e-9943-639595a9b084" (UID: "99474cac-4e87-491e-9943-639595a9b084"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:14:18.099992 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:18.099705 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99474cac-4e87-491e-9943-639595a9b084-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "99474cac-4e87-491e-9943-639595a9b084" (UID: "99474cac-4e87-491e-9943-639595a9b084"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 12:14:18.099992 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:18.099903 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99474cac-4e87-491e-9943-639595a9b084-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "99474cac-4e87-491e-9943-639595a9b084" (UID: "99474cac-4e87-491e-9943-639595a9b084"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:14:18.100091 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:18.100007 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99474cac-4e87-491e-9943-639595a9b084-credential-socket\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:14:18.100091 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:18.100028 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99474cac-4e87-491e-9943-639595a9b084-workload-socket\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:14:18.100091 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:18.100043 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99474cac-4e87-491e-9943-639595a9b084-istiod-ca-cert\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:14:18.100091 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:18.100049 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99474cac-4e87-491e-9943-639595a9b084-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "99474cac-4e87-491e-9943-639595a9b084" (UID: "99474cac-4e87-491e-9943-639595a9b084"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:14:18.100280 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:18.100111 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99474cac-4e87-491e-9943-639595a9b084-istio-data" (OuterVolumeSpecName: "istio-data") pod "99474cac-4e87-491e-9943-639595a9b084" (UID: "99474cac-4e87-491e-9943-639595a9b084"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:14:18.102368 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:18.102343 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99474cac-4e87-491e-9943-639595a9b084-istio-token" (OuterVolumeSpecName: "istio-token") pod "99474cac-4e87-491e-9943-639595a9b084" (UID: "99474cac-4e87-491e-9943-639595a9b084"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:14:18.102368 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:18.102359 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/99474cac-4e87-491e-9943-639595a9b084-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "99474cac-4e87-491e-9943-639595a9b084" (UID: "99474cac-4e87-491e-9943-639595a9b084"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 11 12:14:18.102531 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:18.102339 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99474cac-4e87-491e-9943-639595a9b084-kube-api-access-bnxwr" (OuterVolumeSpecName: "kube-api-access-bnxwr") pod "99474cac-4e87-491e-9943-639595a9b084" (UID: "99474cac-4e87-491e-9943-639595a9b084"). InnerVolumeSpecName "kube-api-access-bnxwr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:14:18.102531 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:18.102404 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99474cac-4e87-491e-9943-639595a9b084-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "99474cac-4e87-491e-9943-639595a9b084" (UID: "99474cac-4e87-491e-9943-639595a9b084"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:14:18.200675 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:18.200630 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99474cac-4e87-491e-9943-639595a9b084-istio-token\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:14:18.200675 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:18.200670 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99474cac-4e87-491e-9943-639595a9b084-istio-podinfo\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:14:18.200675 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:18.200680 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99474cac-4e87-491e-9943-639595a9b084-istio-envoy\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:14:18.200675 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:18.200688 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99474cac-4e87-491e-9943-639595a9b084-istio-data\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:14:18.201064 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:18.200697 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bnxwr\" (UniqueName: \"kubernetes.io/projected/99474cac-4e87-491e-9943-639595a9b084-kube-api-access-bnxwr\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:14:18.201064 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:18.200705 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99474cac-4e87-491e-9943-639595a9b084-workload-certs\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:14:18.912446 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:18.912413 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" event={"ID":"99474cac-4e87-491e-9943-639595a9b084","Type":"ContainerDied","Data":"4a6f8749e04823aef451324a7c4adf686cbb635f310ccfd56922409b103ba011"} Sep 11 12:14:18.912849 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:18.912461 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx" Sep 11 12:14:18.912849 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:18.912475 2566 scope.go:117] "RemoveContainer" containerID="34fb4b8dfe46e0dd55307c8f0345e52bd938883da9c099f176354c7100780ecb" Sep 11 12:14:18.953813 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:18.953781 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx"] Sep 11 12:14:18.958324 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:18.958300 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-f4d189-openshift-default-596bf75ffd-fjjhx"] Sep 11 12:14:19.865833 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:19.865801 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-bf9d5c-7c46789f87-94c7j" Sep 11 12:14:20.758799 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:20.758768 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99474cac-4e87-491e-9943-639595a9b084" path="/var/lib/kubelet/pods/99474cac-4e87-491e-9943-639595a9b084/volumes" Sep 11 12:14:23.534824 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:23.534792 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-45352b-795454fb55-wh6kd"] Sep 11 12:14:23.535248 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:23.535025 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-45352b-795454fb55-wh6kd" podUID="503e63a3-d0bb-4852-8032-ef70c35dd673" containerName="payload-processing" containerID="cri-o://2926bb220d7a932bc39af6b9d01245a68985f523152cd2e24b50d335d3b5bec1" gracePeriod=30 Sep 11 12:14:23.789304 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:23.789241 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-45352b-795454fb55-wh6kd" Sep 11 12:14:23.940751 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:23.940709 2566 generic.go:356] "Generic (PLEG): container finished" podID="503e63a3-d0bb-4852-8032-ef70c35dd673" containerID="2926bb220d7a932bc39af6b9d01245a68985f523152cd2e24b50d335d3b5bec1" exitCode=0 Sep 11 12:14:23.940966 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:23.940758 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-45352b-795454fb55-wh6kd" event={"ID":"503e63a3-d0bb-4852-8032-ef70c35dd673","Type":"ContainerDied","Data":"2926bb220d7a932bc39af6b9d01245a68985f523152cd2e24b50d335d3b5bec1"} Sep 11 12:14:23.940966 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:23.940773 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-45352b-795454fb55-wh6kd" Sep 11 12:14:23.940966 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:23.940835 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-45352b-795454fb55-wh6kd" event={"ID":"503e63a3-d0bb-4852-8032-ef70c35dd673","Type":"ContainerDied","Data":"b4e79451d5e95b18bbd993c1e1d007451038714e64a244e2786f4044fef997fc"} Sep 11 12:14:23.940966 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:23.940859 2566 scope.go:117] "RemoveContainer" containerID="2926bb220d7a932bc39af6b9d01245a68985f523152cd2e24b50d335d3b5bec1" Sep 11 12:14:23.949333 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:23.949311 2566 scope.go:117] "RemoveContainer" containerID="2926bb220d7a932bc39af6b9d01245a68985f523152cd2e24b50d335d3b5bec1" Sep 11 12:14:23.949594 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:14:23.949574 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2926bb220d7a932bc39af6b9d01245a68985f523152cd2e24b50d335d3b5bec1\": container with ID starting with 2926bb220d7a932bc39af6b9d01245a68985f523152cd2e24b50d335d3b5bec1 not found: ID does not exist" containerID="2926bb220d7a932bc39af6b9d01245a68985f523152cd2e24b50d335d3b5bec1" Sep 11 12:14:23.949648 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:23.949605 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2926bb220d7a932bc39af6b9d01245a68985f523152cd2e24b50d335d3b5bec1"} err="failed to get container status \"2926bb220d7a932bc39af6b9d01245a68985f523152cd2e24b50d335d3b5bec1\": rpc error: code = NotFound desc = could not find container \"2926bb220d7a932bc39af6b9d01245a68985f523152cd2e24b50d335d3b5bec1\": container with ID starting with 2926bb220d7a932bc39af6b9d01245a68985f523152cd2e24b50d335d3b5bec1 not found: ID does not exist" Sep 11 12:14:23.952924 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:23.952903 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/503e63a3-d0bb-4852-8032-ef70c35dd673-plugins-config-volume\") pod \"503e63a3-d0bb-4852-8032-ef70c35dd673\" (UID: \"503e63a3-d0bb-4852-8032-ef70c35dd673\") " Sep 11 12:14:23.953039 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:23.953000 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r7l6f\" (UniqueName: \"kubernetes.io/projected/503e63a3-d0bb-4852-8032-ef70c35dd673-kube-api-access-r7l6f\") pod \"503e63a3-d0bb-4852-8032-ef70c35dd673\" (UID: \"503e63a3-d0bb-4852-8032-ef70c35dd673\") " Sep 11 12:14:23.953268 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:23.953244 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/503e63a3-d0bb-4852-8032-ef70c35dd673-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "503e63a3-d0bb-4852-8032-ef70c35dd673" (UID: "503e63a3-d0bb-4852-8032-ef70c35dd673"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 12:14:23.954856 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:23.954834 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/503e63a3-d0bb-4852-8032-ef70c35dd673-kube-api-access-r7l6f" (OuterVolumeSpecName: "kube-api-access-r7l6f") pod "503e63a3-d0bb-4852-8032-ef70c35dd673" (UID: "503e63a3-d0bb-4852-8032-ef70c35dd673"). InnerVolumeSpecName "kube-api-access-r7l6f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:14:24.054361 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:24.054275 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7l6f\" (UniqueName: \"kubernetes.io/projected/503e63a3-d0bb-4852-8032-ef70c35dd673-kube-api-access-r7l6f\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:14:24.054361 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:24.054308 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/503e63a3-d0bb-4852-8032-ef70c35dd673-plugins-config-volume\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:14:24.269696 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:24.269662 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-45352b-795454fb55-wh6kd"] Sep 11 12:14:24.273493 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:24.273468 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-45352b-795454fb55-wh6kd"] Sep 11 12:14:24.759085 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:24.759048 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="503e63a3-d0bb-4852-8032-ef70c35dd673" path="/var/lib/kubelet/pods/503e63a3-d0bb-4852-8032-ef70c35dd673/volumes" Sep 11 12:14:31.453377 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.453343 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7"] Sep 11 12:14:31.453735 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.453689 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99474cac-4e87-491e-9943-639595a9b084" containerName="istio-proxy" Sep 11 12:14:31.453735 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.453700 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="99474cac-4e87-491e-9943-639595a9b084" containerName="istio-proxy" Sep 11 12:14:31.453735 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.453716 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99474cac-4e87-491e-9943-639595a9b084" containerName="istio-proxy" Sep 11 12:14:31.453735 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.453721 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="99474cac-4e87-491e-9943-639595a9b084" containerName="istio-proxy" Sep 11 12:14:31.453735 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.453729 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="503e63a3-d0bb-4852-8032-ef70c35dd673" containerName="payload-processing" Sep 11 12:14:31.453735 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.453735 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="503e63a3-d0bb-4852-8032-ef70c35dd673" containerName="payload-processing" Sep 11 12:14:31.453955 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.453788 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="99474cac-4e87-491e-9943-639595a9b084" containerName="istio-proxy" Sep 11 12:14:31.453955 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.453796 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="99474cac-4e87-491e-9943-639595a9b084" containerName="istio-proxy" Sep 11 12:14:31.453955 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.453804 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="503e63a3-d0bb-4852-8032-ef70c35dd673" containerName="payload-processing" Sep 11 12:14:31.458143 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.458121 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" Sep 11 12:14:31.461020 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.460995 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-986cf2-openshift-default-dockercfg-qp79q\"" Sep 11 12:14:31.470559 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.470537 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7"] Sep 11 12:14:31.512917 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.512858 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6dd23c7-c2ad-4620-a40e-de06161ee314-workload-socket\") pod \"e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7\" (UID: \"b6dd23c7-c2ad-4620-a40e-de06161ee314\") " pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" Sep 11 12:14:31.513083 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.512924 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6dd23c7-c2ad-4620-a40e-de06161ee314-istio-token\") pod \"e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7\" (UID: \"b6dd23c7-c2ad-4620-a40e-de06161ee314\") " pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" Sep 11 12:14:31.513083 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.512966 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6dd23c7-c2ad-4620-a40e-de06161ee314-istio-data\") pod \"e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7\" (UID: \"b6dd23c7-c2ad-4620-a40e-de06161ee314\") " pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" Sep 11 12:14:31.513083 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.512992 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6dd23c7-c2ad-4620-a40e-de06161ee314-istiod-ca-cert\") pod \"e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7\" (UID: \"b6dd23c7-c2ad-4620-a40e-de06161ee314\") " pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" Sep 11 12:14:31.513083 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.513017 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6dd23c7-c2ad-4620-a40e-de06161ee314-istio-podinfo\") pod \"e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7\" (UID: \"b6dd23c7-c2ad-4620-a40e-de06161ee314\") " pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" Sep 11 12:14:31.513083 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.513051 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6dd23c7-c2ad-4620-a40e-de06161ee314-workload-certs\") pod \"e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7\" (UID: \"b6dd23c7-c2ad-4620-a40e-de06161ee314\") " pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" Sep 11 12:14:31.513291 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.513088 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6dd23c7-c2ad-4620-a40e-de06161ee314-istio-envoy\") pod \"e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7\" (UID: \"b6dd23c7-c2ad-4620-a40e-de06161ee314\") " pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" Sep 11 12:14:31.513291 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.513127 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6dd23c7-c2ad-4620-a40e-de06161ee314-credential-socket\") pod \"e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7\" (UID: \"b6dd23c7-c2ad-4620-a40e-de06161ee314\") " pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" Sep 11 12:14:31.513291 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.513162 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68kq9\" (UniqueName: \"kubernetes.io/projected/b6dd23c7-c2ad-4620-a40e-de06161ee314-kube-api-access-68kq9\") pod \"e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7\" (UID: \"b6dd23c7-c2ad-4620-a40e-de06161ee314\") " pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" Sep 11 12:14:31.613952 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.613911 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6dd23c7-c2ad-4620-a40e-de06161ee314-credential-socket\") pod \"e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7\" (UID: \"b6dd23c7-c2ad-4620-a40e-de06161ee314\") " pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" Sep 11 12:14:31.614147 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.613960 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68kq9\" (UniqueName: \"kubernetes.io/projected/b6dd23c7-c2ad-4620-a40e-de06161ee314-kube-api-access-68kq9\") pod \"e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7\" (UID: \"b6dd23c7-c2ad-4620-a40e-de06161ee314\") " pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" Sep 11 12:14:31.614147 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.614024 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6dd23c7-c2ad-4620-a40e-de06161ee314-workload-socket\") pod \"e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7\" (UID: \"b6dd23c7-c2ad-4620-a40e-de06161ee314\") " pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" Sep 11 12:14:31.614147 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.614045 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6dd23c7-c2ad-4620-a40e-de06161ee314-istio-token\") pod \"e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7\" (UID: \"b6dd23c7-c2ad-4620-a40e-de06161ee314\") " pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" Sep 11 12:14:31.614147 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.614087 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6dd23c7-c2ad-4620-a40e-de06161ee314-istio-data\") pod \"e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7\" (UID: \"b6dd23c7-c2ad-4620-a40e-de06161ee314\") " pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" Sep 11 12:14:31.614147 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.614111 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6dd23c7-c2ad-4620-a40e-de06161ee314-istiod-ca-cert\") pod \"e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7\" (UID: \"b6dd23c7-c2ad-4620-a40e-de06161ee314\") " pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" Sep 11 12:14:31.614436 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.614160 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6dd23c7-c2ad-4620-a40e-de06161ee314-istio-podinfo\") pod \"e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7\" (UID: \"b6dd23c7-c2ad-4620-a40e-de06161ee314\") " pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" Sep 11 12:14:31.614436 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.614201 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6dd23c7-c2ad-4620-a40e-de06161ee314-workload-certs\") pod \"e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7\" (UID: \"b6dd23c7-c2ad-4620-a40e-de06161ee314\") " pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" Sep 11 12:14:31.614436 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.614233 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6dd23c7-c2ad-4620-a40e-de06161ee314-istio-envoy\") pod \"e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7\" (UID: \"b6dd23c7-c2ad-4620-a40e-de06161ee314\") " pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" Sep 11 12:14:31.614602 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.614510 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6dd23c7-c2ad-4620-a40e-de06161ee314-istio-data\") pod \"e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7\" (UID: \"b6dd23c7-c2ad-4620-a40e-de06161ee314\") " pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" Sep 11 12:14:31.614602 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.614562 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6dd23c7-c2ad-4620-a40e-de06161ee314-workload-certs\") pod \"e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7\" (UID: \"b6dd23c7-c2ad-4620-a40e-de06161ee314\") " pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" Sep 11 12:14:31.614719 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.614701 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6dd23c7-c2ad-4620-a40e-de06161ee314-workload-socket\") pod \"e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7\" (UID: \"b6dd23c7-c2ad-4620-a40e-de06161ee314\") " pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" Sep 11 12:14:31.614904 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.614853 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6dd23c7-c2ad-4620-a40e-de06161ee314-credential-socket\") pod \"e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7\" (UID: \"b6dd23c7-c2ad-4620-a40e-de06161ee314\") " pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" Sep 11 12:14:31.614904 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.614855 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6dd23c7-c2ad-4620-a40e-de06161ee314-istiod-ca-cert\") pod \"e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7\" (UID: \"b6dd23c7-c2ad-4620-a40e-de06161ee314\") " pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" Sep 11 12:14:31.617063 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.617041 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6dd23c7-c2ad-4620-a40e-de06161ee314-istio-envoy\") pod \"e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7\" (UID: \"b6dd23c7-c2ad-4620-a40e-de06161ee314\") " pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" Sep 11 12:14:31.617177 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.617160 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6dd23c7-c2ad-4620-a40e-de06161ee314-istio-podinfo\") pod \"e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7\" (UID: \"b6dd23c7-c2ad-4620-a40e-de06161ee314\") " pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" Sep 11 12:14:31.624185 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.624156 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6dd23c7-c2ad-4620-a40e-de06161ee314-istio-token\") pod \"e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7\" (UID: \"b6dd23c7-c2ad-4620-a40e-de06161ee314\") " pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" Sep 11 12:14:31.624685 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.624663 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68kq9\" (UniqueName: \"kubernetes.io/projected/b6dd23c7-c2ad-4620-a40e-de06161ee314-kube-api-access-68kq9\") pod \"e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7\" (UID: \"b6dd23c7-c2ad-4620-a40e-de06161ee314\") " pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" Sep 11 12:14:31.767764 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:31.767678 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" Sep 11 12:14:32.044596 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:32.044572 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7"] Sep 11 12:14:32.047178 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:14:32.047149 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6dd23c7_c2ad_4620_a40e_de06161ee314.slice/crio-dea80391550c7853b1298ac81532c39be906e1b163148f436ce663af6298e86b WatchSource:0}: Error finding container dea80391550c7853b1298ac81532c39be906e1b163148f436ce663af6298e86b: Status 404 returned error can't find the container with id dea80391550c7853b1298ac81532c39be906e1b163148f436ce663af6298e86b Sep 11 12:14:32.049598 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:32.049566 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:14:32.049706 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:32.049632 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:14:32.049706 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:32.049666 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:14:32.989031 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:32.988996 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" event={"ID":"b6dd23c7-c2ad-4620-a40e-de06161ee314","Type":"ContainerStarted","Data":"b6696c33cfec5fd5b4a3513e0a74e94fc70a2639f38deb7001d9ccdb3820805e"} Sep 11 12:14:32.989395 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:32.989049 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" event={"ID":"b6dd23c7-c2ad-4620-a40e-de06161ee314","Type":"ContainerStarted","Data":"dea80391550c7853b1298ac81532c39be906e1b163148f436ce663af6298e86b"} Sep 11 12:14:33.017733 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:33.017681 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" podStartSLOduration=2.017665855 podStartE2EDuration="2.017665855s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:14:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:14:33.01609083 +0000 UTC m=+1604.781064006" watchObservedRunningTime="2026-09-11 12:14:33.017665855 +0000 UTC m=+1604.782639029" Sep 11 12:14:33.768582 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:33.768549 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" Sep 11 12:14:33.776143 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:33.776109 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" Sep 11 12:14:33.994403 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:33.994368 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" Sep 11 12:14:33.995257 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:33.995236 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" Sep 11 12:14:42.285606 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:42.285519 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7851b-69678bc8b6-rg8fq"] Sep 11 12:14:42.286133 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:42.285786 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7851b-69678bc8b6-rg8fq" podUID="6b3221af-e568-460b-88fa-a0f1151c3772" containerName="payload-pre-processing" containerID="cri-o://c9180437c8dd8fc3b04f786bb272e65397934c3877890588639500abb2858655" gracePeriod=30 Sep 11 12:14:42.537423 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:42.537356 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7851b-69678bc8b6-rg8fq" Sep 11 12:14:42.598959 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:42.598928 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vx4q7\" (UniqueName: \"kubernetes.io/projected/6b3221af-e568-460b-88fa-a0f1151c3772-kube-api-access-vx4q7\") pod \"6b3221af-e568-460b-88fa-a0f1151c3772\" (UID: \"6b3221af-e568-460b-88fa-a0f1151c3772\") " Sep 11 12:14:42.599153 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:42.599045 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b3221af-e568-460b-88fa-a0f1151c3772-plugins-config-volume\") pod \"6b3221af-e568-460b-88fa-a0f1151c3772\" (UID: \"6b3221af-e568-460b-88fa-a0f1151c3772\") " Sep 11 12:14:42.599426 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:42.599397 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b3221af-e568-460b-88fa-a0f1151c3772-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6b3221af-e568-460b-88fa-a0f1151c3772" (UID: "6b3221af-e568-460b-88fa-a0f1151c3772"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 12:14:42.601092 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:42.601066 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b3221af-e568-460b-88fa-a0f1151c3772-kube-api-access-vx4q7" (OuterVolumeSpecName: "kube-api-access-vx4q7") pod "6b3221af-e568-460b-88fa-a0f1151c3772" (UID: "6b3221af-e568-460b-88fa-a0f1151c3772"). InnerVolumeSpecName "kube-api-access-vx4q7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:14:42.700610 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:42.700575 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b3221af-e568-460b-88fa-a0f1151c3772-plugins-config-volume\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:14:42.700610 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:42.700602 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vx4q7\" (UniqueName: \"kubernetes.io/projected/6b3221af-e568-460b-88fa-a0f1151c3772-kube-api-access-vx4q7\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:14:43.040234 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:43.040191 2566 generic.go:356] "Generic (PLEG): container finished" podID="6b3221af-e568-460b-88fa-a0f1151c3772" containerID="c9180437c8dd8fc3b04f786bb272e65397934c3877890588639500abb2858655" exitCode=0 Sep 11 12:14:43.040425 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:43.040242 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7851b-69678bc8b6-rg8fq" Sep 11 12:14:43.040425 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:43.040283 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7851b-69678bc8b6-rg8fq" event={"ID":"6b3221af-e568-460b-88fa-a0f1151c3772","Type":"ContainerDied","Data":"c9180437c8dd8fc3b04f786bb272e65397934c3877890588639500abb2858655"} Sep 11 12:14:43.040425 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:43.040340 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7851b-69678bc8b6-rg8fq" event={"ID":"6b3221af-e568-460b-88fa-a0f1151c3772","Type":"ContainerDied","Data":"aa78eb0cb6e68cdc3a7f136a7cfdea0dc1d41e30b618e1207f96d1b8ee49380a"} Sep 11 12:14:43.040425 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:43.040363 2566 scope.go:117] "RemoveContainer" containerID="c9180437c8dd8fc3b04f786bb272e65397934c3877890588639500abb2858655" Sep 11 12:14:43.050107 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:43.050088 2566 scope.go:117] "RemoveContainer" containerID="c9180437c8dd8fc3b04f786bb272e65397934c3877890588639500abb2858655" Sep 11 12:14:43.050416 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:14:43.050397 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9180437c8dd8fc3b04f786bb272e65397934c3877890588639500abb2858655\": container with ID starting with c9180437c8dd8fc3b04f786bb272e65397934c3877890588639500abb2858655 not found: ID does not exist" containerID="c9180437c8dd8fc3b04f786bb272e65397934c3877890588639500abb2858655" Sep 11 12:14:43.050519 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:43.050421 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9180437c8dd8fc3b04f786bb272e65397934c3877890588639500abb2858655"} err="failed to get container status \"c9180437c8dd8fc3b04f786bb272e65397934c3877890588639500abb2858655\": rpc error: code = NotFound desc = could not find container \"c9180437c8dd8fc3b04f786bb272e65397934c3877890588639500abb2858655\": container with ID starting with c9180437c8dd8fc3b04f786bb272e65397934c3877890588639500abb2858655 not found: ID does not exist" Sep 11 12:14:43.059690 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:43.059665 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7851b-69678bc8b6-rg8fq"] Sep 11 12:14:43.063752 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:43.063731 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7851b-69678bc8b6-rg8fq"] Sep 11 12:14:44.758568 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:44.758534 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b3221af-e568-460b-88fa-a0f1151c3772" path="/var/lib/kubelet/pods/6b3221af-e568-460b-88fa-a0f1151c3772/volumes" Sep 11 12:14:49.630329 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:49.630291 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58"] Sep 11 12:14:49.630809 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:49.630489 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" podUID="f7a938ed-a372-4280-84a8-33877f1fd3be" containerName="istio-proxy" containerID="cri-o://108b966e56cbe0a7fc527c58481d455d61eb206fcb948c0869c0f48b4eed9a05" gracePeriod=30 Sep 11 12:14:52.088791 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:52.088703 2566 generic.go:356] "Generic (PLEG): container finished" podID="b6dd23c7-c2ad-4620-a40e-de06161ee314" containerID="b6696c33cfec5fd5b4a3513e0a74e94fc70a2639f38deb7001d9ccdb3820805e" exitCode=0 Sep 11 12:14:52.088791 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:52.088775 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" event={"ID":"b6dd23c7-c2ad-4620-a40e-de06161ee314","Type":"ContainerDied","Data":"b6696c33cfec5fd5b4a3513e0a74e94fc70a2639f38deb7001d9ccdb3820805e"} Sep 11 12:14:52.089242 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:52.089187 2566 scope.go:117] "RemoveContainer" containerID="b6696c33cfec5fd5b4a3513e0a74e94fc70a2639f38deb7001d9ccdb3820805e" Sep 11 12:14:52.090277 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:52.090244 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:14:52.090385 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:52.090327 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:14:52.090385 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:52.090373 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:14:53.095255 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:53.095221 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" event={"ID":"b6dd23c7-c2ad-4620-a40e-de06161ee314","Type":"ContainerStarted","Data":"30705e049347ed93c08caa39aeb3f83e19a0cc8dd3ca3241a5c130c4f03f7e05"} Sep 11 12:14:53.768100 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:53.768063 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" Sep 11 12:14:53.768283 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:53.768199 2566 patch_prober.go:28] interesting pod/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Sep 11 12:14:53.768283 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:53.768250 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" podUID="b6dd23c7-c2ad-4620-a40e-de06161ee314" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Sep 11 12:14:54.768131 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:54.768092 2566 patch_prober.go:28] interesting pod/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Sep 11 12:14:54.768474 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:54.768157 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" podUID="b6dd23c7-c2ad-4620-a40e-de06161ee314" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Sep 11 12:14:54.892322 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:54.892266 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" Sep 11 12:14:55.028947 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.028912 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7a938ed-a372-4280-84a8-33877f1fd3be-istio-token\") pod \"f7a938ed-a372-4280-84a8-33877f1fd3be\" (UID: \"f7a938ed-a372-4280-84a8-33877f1fd3be\") " Sep 11 12:14:55.029115 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.028979 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7a938ed-a372-4280-84a8-33877f1fd3be-credential-socket\") pod \"f7a938ed-a372-4280-84a8-33877f1fd3be\" (UID: \"f7a938ed-a372-4280-84a8-33877f1fd3be\") " Sep 11 12:14:55.029115 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.029017 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7a938ed-a372-4280-84a8-33877f1fd3be-istiod-ca-cert\") pod \"f7a938ed-a372-4280-84a8-33877f1fd3be\" (UID: \"f7a938ed-a372-4280-84a8-33877f1fd3be\") " Sep 11 12:14:55.029115 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.029035 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7a938ed-a372-4280-84a8-33877f1fd3be-istio-envoy\") pod \"f7a938ed-a372-4280-84a8-33877f1fd3be\" (UID: \"f7a938ed-a372-4280-84a8-33877f1fd3be\") " Sep 11 12:14:55.029115 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.029051 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7a938ed-a372-4280-84a8-33877f1fd3be-workload-socket\") pod \"f7a938ed-a372-4280-84a8-33877f1fd3be\" (UID: \"f7a938ed-a372-4280-84a8-33877f1fd3be\") " Sep 11 12:14:55.029115 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.029084 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7a938ed-a372-4280-84a8-33877f1fd3be-istio-data\") pod \"f7a938ed-a372-4280-84a8-33877f1fd3be\" (UID: \"f7a938ed-a372-4280-84a8-33877f1fd3be\") " Sep 11 12:14:55.029287 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.029132 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7a938ed-a372-4280-84a8-33877f1fd3be-istio-podinfo\") pod \"f7a938ed-a372-4280-84a8-33877f1fd3be\" (UID: \"f7a938ed-a372-4280-84a8-33877f1fd3be\") " Sep 11 12:14:55.029287 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.029146 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7a938ed-a372-4280-84a8-33877f1fd3be-workload-certs\") pod \"f7a938ed-a372-4280-84a8-33877f1fd3be\" (UID: \"f7a938ed-a372-4280-84a8-33877f1fd3be\") " Sep 11 12:14:55.029287 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.029165 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mtgfv\" (UniqueName: \"kubernetes.io/projected/f7a938ed-a372-4280-84a8-33877f1fd3be-kube-api-access-mtgfv\") pod \"f7a938ed-a372-4280-84a8-33877f1fd3be\" (UID: \"f7a938ed-a372-4280-84a8-33877f1fd3be\") " Sep 11 12:14:55.029287 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.029166 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7a938ed-a372-4280-84a8-33877f1fd3be-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f7a938ed-a372-4280-84a8-33877f1fd3be" (UID: "f7a938ed-a372-4280-84a8-33877f1fd3be"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:14:55.029464 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.029363 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7a938ed-a372-4280-84a8-33877f1fd3be-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f7a938ed-a372-4280-84a8-33877f1fd3be" (UID: "f7a938ed-a372-4280-84a8-33877f1fd3be"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 12:14:55.029464 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.029437 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7a938ed-a372-4280-84a8-33877f1fd3be-istio-data" (OuterVolumeSpecName: "istio-data") pod "f7a938ed-a372-4280-84a8-33877f1fd3be" (UID: "f7a938ed-a372-4280-84a8-33877f1fd3be"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:14:55.029566 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.029528 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7a938ed-a372-4280-84a8-33877f1fd3be-istiod-ca-cert\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:14:55.029566 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.029548 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7a938ed-a372-4280-84a8-33877f1fd3be-istio-data\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:14:55.029566 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.029562 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7a938ed-a372-4280-84a8-33877f1fd3be-credential-socket\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:14:55.029721 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.029564 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7a938ed-a372-4280-84a8-33877f1fd3be-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f7a938ed-a372-4280-84a8-33877f1fd3be" (UID: "f7a938ed-a372-4280-84a8-33877f1fd3be"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:14:55.029776 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.029713 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7a938ed-a372-4280-84a8-33877f1fd3be-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f7a938ed-a372-4280-84a8-33877f1fd3be" (UID: "f7a938ed-a372-4280-84a8-33877f1fd3be"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:14:55.031269 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.031241 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7a938ed-a372-4280-84a8-33877f1fd3be-istio-token" (OuterVolumeSpecName: "istio-token") pod "f7a938ed-a372-4280-84a8-33877f1fd3be" (UID: "f7a938ed-a372-4280-84a8-33877f1fd3be"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:14:55.031574 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.031552 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7a938ed-a372-4280-84a8-33877f1fd3be-kube-api-access-mtgfv" (OuterVolumeSpecName: "kube-api-access-mtgfv") pod "f7a938ed-a372-4280-84a8-33877f1fd3be" (UID: "f7a938ed-a372-4280-84a8-33877f1fd3be"). InnerVolumeSpecName "kube-api-access-mtgfv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:14:55.031746 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.031723 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f7a938ed-a372-4280-84a8-33877f1fd3be-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f7a938ed-a372-4280-84a8-33877f1fd3be" (UID: "f7a938ed-a372-4280-84a8-33877f1fd3be"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 11 12:14:55.031917 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.031897 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7a938ed-a372-4280-84a8-33877f1fd3be-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f7a938ed-a372-4280-84a8-33877f1fd3be" (UID: "f7a938ed-a372-4280-84a8-33877f1fd3be"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:14:55.114590 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.114542 2566 generic.go:356] "Generic (PLEG): container finished" podID="f7a938ed-a372-4280-84a8-33877f1fd3be" containerID="108b966e56cbe0a7fc527c58481d455d61eb206fcb948c0869c0f48b4eed9a05" exitCode=0 Sep 11 12:14:55.114779 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.114641 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" event={"ID":"f7a938ed-a372-4280-84a8-33877f1fd3be","Type":"ContainerDied","Data":"108b966e56cbe0a7fc527c58481d455d61eb206fcb948c0869c0f48b4eed9a05"} Sep 11 12:14:55.114779 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.114691 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" event={"ID":"f7a938ed-a372-4280-84a8-33877f1fd3be","Type":"ContainerDied","Data":"8ccbc87b00ad977b805aaea8d88f2509d26fbe41259583ab7f56ae1c34dd4bc9"} Sep 11 12:14:55.114779 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.114719 2566 scope.go:117] "RemoveContainer" containerID="108b966e56cbe0a7fc527c58481d455d61eb206fcb948c0869c0f48b4eed9a05" Sep 11 12:14:55.114982 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.114914 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58" Sep 11 12:14:55.130836 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.130811 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7a938ed-a372-4280-84a8-33877f1fd3be-istio-token\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:14:55.130978 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.130954 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7a938ed-a372-4280-84a8-33877f1fd3be-istio-envoy\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:14:55.130978 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.130974 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7a938ed-a372-4280-84a8-33877f1fd3be-workload-socket\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:14:55.131085 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.130990 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7a938ed-a372-4280-84a8-33877f1fd3be-istio-podinfo\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:14:55.131085 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.131004 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7a938ed-a372-4280-84a8-33877f1fd3be-workload-certs\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:14:55.131085 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.131018 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mtgfv\" (UniqueName: \"kubernetes.io/projected/f7a938ed-a372-4280-84a8-33877f1fd3be-kube-api-access-mtgfv\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:14:55.134227 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.134203 2566 scope.go:117] "RemoveContainer" containerID="ebe06adafe28318f35f81150bffc1e25e6c9ce723a493c95946f04b51088d1be" Sep 11 12:14:55.142418 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.142378 2566 scope.go:117] "RemoveContainer" containerID="108b966e56cbe0a7fc527c58481d455d61eb206fcb948c0869c0f48b4eed9a05" Sep 11 12:14:55.142646 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:14:55.142630 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"108b966e56cbe0a7fc527c58481d455d61eb206fcb948c0869c0f48b4eed9a05\": container with ID starting with 108b966e56cbe0a7fc527c58481d455d61eb206fcb948c0869c0f48b4eed9a05 not found: ID does not exist" containerID="108b966e56cbe0a7fc527c58481d455d61eb206fcb948c0869c0f48b4eed9a05" Sep 11 12:14:55.142699 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.142652 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"108b966e56cbe0a7fc527c58481d455d61eb206fcb948c0869c0f48b4eed9a05"} err="failed to get container status \"108b966e56cbe0a7fc527c58481d455d61eb206fcb948c0869c0f48b4eed9a05\": rpc error: code = NotFound desc = could not find container \"108b966e56cbe0a7fc527c58481d455d61eb206fcb948c0869c0f48b4eed9a05\": container with ID starting with 108b966e56cbe0a7fc527c58481d455d61eb206fcb948c0869c0f48b4eed9a05 not found: ID does not exist" Sep 11 12:14:55.142699 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.142668 2566 scope.go:117] "RemoveContainer" containerID="ebe06adafe28318f35f81150bffc1e25e6c9ce723a493c95946f04b51088d1be" Sep 11 12:14:55.142990 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:14:55.142975 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebe06adafe28318f35f81150bffc1e25e6c9ce723a493c95946f04b51088d1be\": container with ID starting with ebe06adafe28318f35f81150bffc1e25e6c9ce723a493c95946f04b51088d1be not found: ID does not exist" containerID="ebe06adafe28318f35f81150bffc1e25e6c9ce723a493c95946f04b51088d1be" Sep 11 12:14:55.143040 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.142994 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebe06adafe28318f35f81150bffc1e25e6c9ce723a493c95946f04b51088d1be"} err="failed to get container status \"ebe06adafe28318f35f81150bffc1e25e6c9ce723a493c95946f04b51088d1be\": rpc error: code = NotFound desc = could not find container \"ebe06adafe28318f35f81150bffc1e25e6c9ce723a493c95946f04b51088d1be\": container with ID starting with ebe06adafe28318f35f81150bffc1e25e6c9ce723a493c95946f04b51088d1be not found: ID does not exist" Sep 11 12:14:55.149421 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.149398 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58"] Sep 11 12:14:55.156329 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.156306 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-f7851b-openshift-default-6f6578d9cd-zwb58"] Sep 11 12:14:55.768610 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.768567 2566 patch_prober.go:28] interesting pod/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Sep 11 12:14:55.769045 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:55.768642 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" podUID="b6dd23c7-c2ad-4620-a40e-de06161ee314" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Sep 11 12:14:56.758347 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:56.758314 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7a938ed-a372-4280-84a8-33877f1fd3be" path="/var/lib/kubelet/pods/f7a938ed-a372-4280-84a8-33877f1fd3be/volumes" Sep 11 12:14:56.768529 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:56.768504 2566 patch_prober.go:28] interesting pod/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Sep 11 12:14:56.768678 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:56.768570 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" podUID="b6dd23c7-c2ad-4620-a40e-de06161ee314" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Sep 11 12:14:57.768752 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:57.768715 2566 patch_prober.go:28] interesting pod/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Sep 11 12:14:57.769250 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:57.768781 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" podUID="b6dd23c7-c2ad-4620-a40e-de06161ee314" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Sep 11 12:14:58.772411 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:58.772380 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" Sep 11 12:14:58.772791 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:58.772652 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" Sep 11 12:14:58.773343 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:14:58.773320 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" Sep 11 12:15:00.194475 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:00.194441 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-af081c-29818815-bfgnh"] Sep 11 12:15:00.194896 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:00.194802 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7a938ed-a372-4280-84a8-33877f1fd3be" containerName="istio-proxy" Sep 11 12:15:00.194896 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:00.194815 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7a938ed-a372-4280-84a8-33877f1fd3be" containerName="istio-proxy" Sep 11 12:15:00.194896 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:00.194836 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b3221af-e568-460b-88fa-a0f1151c3772" containerName="payload-pre-processing" Sep 11 12:15:00.194896 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:00.194842 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b3221af-e568-460b-88fa-a0f1151c3772" containerName="payload-pre-processing" Sep 11 12:15:00.195042 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:00.194922 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7a938ed-a372-4280-84a8-33877f1fd3be" containerName="istio-proxy" Sep 11 12:15:00.195042 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:00.194934 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b3221af-e568-460b-88fa-a0f1151c3772" containerName="payload-pre-processing" Sep 11 12:15:00.195042 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:00.194940 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7a938ed-a372-4280-84a8-33877f1fd3be" containerName="istio-proxy" Sep 11 12:15:00.199235 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:00.199214 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29818815-ww8b8"] Sep 11 12:15:00.199343 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:00.199324 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-af081c-29818815-bfgnh" Sep 11 12:15:00.199542 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:00.199520 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7a938ed-a372-4280-84a8-33877f1fd3be" containerName="istio-proxy" Sep 11 12:15:00.199542 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:00.199538 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7a938ed-a372-4280-84a8-33877f1fd3be" containerName="istio-proxy" Sep 11 12:15:00.202280 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:00.202259 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818815-ww8b8" Sep 11 12:15:00.216913 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:00.216857 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9xkpf\"" Sep 11 12:15:00.217485 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:00.217464 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818815-ww8b8" Sep 11 12:15:00.219374 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:00.219348 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-af081c-29818815-bfgnh"] Sep 11 12:15:00.220115 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:00.220094 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-af081c-29818815-bfgnh" Sep 11 12:15:00.221764 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:00.221740 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29818815-ww8b8"] Sep 11 12:15:00.496267 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:00.496237 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29818815-ww8b8"] Sep 11 12:15:00.497787 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:15:00.497760 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c476fe9_7d91_4ae5_bf84_72180ba00705.slice/crio-15ee9be55ad384d94e764923baf2421b4cb0d0a1a52a5685333e3044c514b591 WatchSource:0}: Error finding container 15ee9be55ad384d94e764923baf2421b4cb0d0a1a52a5685333e3044c514b591: Status 404 returned error can't find the container with id 15ee9be55ad384d94e764923baf2421b4cb0d0a1a52a5685333e3044c514b591 Sep 11 12:15:00.499640 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:00.499620 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 11 12:15:00.504372 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:00.504329 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-af081c-29818815-bfgnh"] Sep 11 12:15:00.505570 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:15:00.505546 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8d35023_c90e_4a2c_9df6_9c67c9ceefa7.slice/crio-7e9b9392c56c275be73e9da00b2a1243eeee4e529895c2dbe765ac71c9f53696 WatchSource:0}: Error finding container 7e9b9392c56c275be73e9da00b2a1243eeee4e529895c2dbe765ac71c9f53696: Status 404 returned error can't find the container with id 7e9b9392c56c275be73e9da00b2a1243eeee4e529895c2dbe765ac71c9f53696 Sep 11 12:15:01.149700 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:01.149662 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-af081c-29818815-bfgnh" event={"ID":"a8d35023-c90e-4a2c-9df6-9c67c9ceefa7","Type":"ContainerStarted","Data":"7e9b9392c56c275be73e9da00b2a1243eeee4e529895c2dbe765ac71c9f53696"} Sep 11 12:15:01.150877 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:01.150835 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818815-ww8b8" event={"ID":"1c476fe9-7d91-4ae5-bf84-72180ba00705","Type":"ContainerStarted","Data":"15ee9be55ad384d94e764923baf2421b4cb0d0a1a52a5685333e3044c514b591"} Sep 11 12:15:04.168227 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:04.168129 2566 generic.go:356] "Generic (PLEG): container finished" podID="a8d35023-c90e-4a2c-9df6-9c67c9ceefa7" containerID="5ff377c17f78f3ada4d1acf5d0799a31db0b34962d3881bc49969e7d6f1d94dc" exitCode=0 Sep 11 12:15:04.168650 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:04.168220 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-af081c-29818815-bfgnh" event={"ID":"a8d35023-c90e-4a2c-9df6-9c67c9ceefa7","Type":"ContainerDied","Data":"5ff377c17f78f3ada4d1acf5d0799a31db0b34962d3881bc49969e7d6f1d94dc"} Sep 11 12:15:04.169651 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:04.169628 2566 generic.go:356] "Generic (PLEG): container finished" podID="1c476fe9-7d91-4ae5-bf84-72180ba00705" containerID="4c21c628450c8f9f951207ea340d09d6b2eff3ced3b8dd7849b28aaebbb9ba6c" exitCode=0 Sep 11 12:15:04.169757 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:04.169666 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818815-ww8b8" event={"ID":"1c476fe9-7d91-4ae5-bf84-72180ba00705","Type":"ContainerDied","Data":"4c21c628450c8f9f951207ea340d09d6b2eff3ced3b8dd7849b28aaebbb9ba6c"} Sep 11 12:15:04.921047 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:04.921016 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-pq4hx_e55f3a24-f1c7-4556-a5eb-8a9ba6e89fe1/payload-processing/0.log" Sep 11 12:15:05.329258 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:05.329234 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-af081c-29818815-bfgnh" Sep 11 12:15:05.332064 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:05.332042 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818815-ww8b8" Sep 11 12:15:06.180593 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:06.180556 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-af081c-29818815-bfgnh" event={"ID":"a8d35023-c90e-4a2c-9df6-9c67c9ceefa7","Type":"ContainerDied","Data":"7e9b9392c56c275be73e9da00b2a1243eeee4e529895c2dbe765ac71c9f53696"} Sep 11 12:15:06.180788 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:06.180607 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7e9b9392c56c275be73e9da00b2a1243eeee4e529895c2dbe765ac71c9f53696" Sep 11 12:15:06.180788 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:06.180564 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-af081c-29818815-bfgnh" Sep 11 12:15:06.181920 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:06.181896 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818815-ww8b8" event={"ID":"1c476fe9-7d91-4ae5-bf84-72180ba00705","Type":"ContainerDied","Data":"15ee9be55ad384d94e764923baf2421b4cb0d0a1a52a5685333e3044c514b591"} Sep 11 12:15:06.182020 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:06.181938 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="15ee9be55ad384d94e764923baf2421b4cb0d0a1a52a5685333e3044c514b591" Sep 11 12:15:06.182020 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:06.181902 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818815-ww8b8" Sep 11 12:15:55.934501 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:55.934466 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-bf9d5c-7c46789f87-94c7j_74844184-6bfc-476f-b940-14bbb7018f24/payload-processing/0.log" Sep 11 12:15:56.060209 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:56.060176 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-pq4hx_e55f3a24-f1c7-4556-a5eb-8a9ba6e89fe1/payload-processing/0.log" Sep 11 12:15:56.720044 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:56.720003 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq"] Sep 11 12:15:56.720614 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:56.720588 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c476fe9-7d91-4ae5-bf84-72180ba00705" containerName="cleanup" Sep 11 12:15:56.720614 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:56.720615 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c476fe9-7d91-4ae5-bf84-72180ba00705" containerName="cleanup" Sep 11 12:15:56.720773 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:56.720639 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8d35023-c90e-4a2c-9df6-9c67c9ceefa7" containerName="cleanup" Sep 11 12:15:56.720773 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:56.720657 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8d35023-c90e-4a2c-9df6-9c67c9ceefa7" containerName="cleanup" Sep 11 12:15:56.720773 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:56.720729 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c476fe9-7d91-4ae5-bf84-72180ba00705" containerName="cleanup" Sep 11 12:15:56.720773 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:56.720743 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8d35023-c90e-4a2c-9df6-9c67c9ceefa7" containerName="cleanup" Sep 11 12:15:56.723017 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:56.722998 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" Sep 11 12:15:56.725973 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:56.725954 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-5c1bd9-openshift-default-dockercfg-mz7l7\"" Sep 11 12:15:56.741070 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:56.741045 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq"] Sep 11 12:15:56.879259 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:56.879219 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-workload-socket\") pod \"e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq\" (UID: \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\") " pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" Sep 11 12:15:56.879533 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:56.879508 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-workload-certs\") pod \"e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq\" (UID: \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\") " pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" Sep 11 12:15:56.879718 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:56.879702 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-istio-envoy\") pod \"e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq\" (UID: \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\") " pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" Sep 11 12:15:56.879948 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:56.879926 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-istiod-ca-cert\") pod \"e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq\" (UID: \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\") " pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" Sep 11 12:15:56.880051 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:56.880027 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-istio-token\") pod \"e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq\" (UID: \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\") " pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" Sep 11 12:15:56.880113 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:56.880078 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-credential-socket\") pod \"e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq\" (UID: \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\") " pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" Sep 11 12:15:56.880173 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:56.880129 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-istio-data\") pod \"e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq\" (UID: \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\") " pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" Sep 11 12:15:56.880224 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:56.880172 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mz5d6\" (UniqueName: \"kubernetes.io/projected/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-kube-api-access-mz5d6\") pod \"e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq\" (UID: \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\") " pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" Sep 11 12:15:56.880286 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:56.880272 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-istio-podinfo\") pod \"e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq\" (UID: \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\") " pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" Sep 11 12:15:56.981303 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:56.981193 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-workload-socket\") pod \"e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq\" (UID: \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\") " pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" Sep 11 12:15:56.981303 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:56.981248 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-workload-certs\") pod \"e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq\" (UID: \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\") " pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" Sep 11 12:15:56.981303 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:56.981273 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-istio-envoy\") pod \"e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq\" (UID: \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\") " pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" Sep 11 12:15:56.981303 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:56.981301 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-istiod-ca-cert\") pod \"e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq\" (UID: \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\") " pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" Sep 11 12:15:56.981916 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:56.981318 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-istio-token\") pod \"e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq\" (UID: \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\") " pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" Sep 11 12:15:56.981916 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:56.981345 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-credential-socket\") pod \"e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq\" (UID: \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\") " pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" Sep 11 12:15:56.981916 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:56.981380 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-istio-data\") pod \"e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq\" (UID: \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\") " pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" Sep 11 12:15:56.981916 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:56.981417 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mz5d6\" (UniqueName: \"kubernetes.io/projected/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-kube-api-access-mz5d6\") pod \"e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq\" (UID: \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\") " pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" Sep 11 12:15:56.981916 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:56.981451 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-istio-podinfo\") pod \"e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq\" (UID: \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\") " pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" Sep 11 12:15:56.981916 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:56.981636 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-workload-socket\") pod \"e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq\" (UID: \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\") " pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" Sep 11 12:15:56.981916 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:56.981681 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-workload-certs\") pod \"e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq\" (UID: \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\") " pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" Sep 11 12:15:56.981916 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:56.981784 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-credential-socket\") pod \"e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq\" (UID: \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\") " pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" Sep 11 12:15:56.982201 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:56.981947 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-istio-data\") pod \"e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq\" (UID: \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\") " pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" Sep 11 12:15:56.982201 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:56.982061 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-istiod-ca-cert\") pod \"e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq\" (UID: \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\") " pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" Sep 11 12:15:56.983671 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:56.983651 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-istio-envoy\") pod \"e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq\" (UID: \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\") " pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" Sep 11 12:15:56.983898 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:56.983879 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-istio-podinfo\") pod \"e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq\" (UID: \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\") " pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" Sep 11 12:15:56.990714 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:56.990686 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-istio-token\") pod \"e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq\" (UID: \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\") " pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" Sep 11 12:15:56.990982 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:56.990960 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mz5d6\" (UniqueName: \"kubernetes.io/projected/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-kube-api-access-mz5d6\") pod \"e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq\" (UID: \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\") " pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" Sep 11 12:15:57.033975 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:57.033940 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" Sep 11 12:15:57.312855 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:57.312824 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq"] Sep 11 12:15:57.315836 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:15:57.315806 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33a4cd0e_67ae_4baa_8551_d1a6c21b2cd0.slice/crio-7ef42982b12e30527c92e1560cd342d06b8d1f94cc2275a623acd90091944515 WatchSource:0}: Error finding container 7ef42982b12e30527c92e1560cd342d06b8d1f94cc2275a623acd90091944515: Status 404 returned error can't find the container with id 7ef42982b12e30527c92e1560cd342d06b8d1f94cc2275a623acd90091944515 Sep 11 12:15:57.318754 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:57.318720 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:15:57.318831 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:57.318795 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:15:57.318831 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:57.318823 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:15:57.430970 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:57.430923 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" event={"ID":"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0","Type":"ContainerStarted","Data":"019abb4bd9402f98aaa526f0c96671d91d557752757b7c9b87743364a593f84e"} Sep 11 12:15:57.431166 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:57.430990 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" event={"ID":"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0","Type":"ContainerStarted","Data":"7ef42982b12e30527c92e1560cd342d06b8d1f94cc2275a623acd90091944515"} Sep 11 12:15:57.459306 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:57.459242 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" podStartSLOduration=1.459222459 podStartE2EDuration="1.459222459s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:15:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:15:57.454186773 +0000 UTC m=+1689.219159950" watchObservedRunningTime="2026-09-11 12:15:57.459222459 +0000 UTC m=+1689.224195635" Sep 11 12:15:58.034300 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:58.034261 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" Sep 11 12:15:58.039838 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:58.039809 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" Sep 11 12:15:58.436314 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:58.436281 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" Sep 11 12:15:58.437158 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:15:58.437138 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" Sep 11 12:16:01.335935 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:01.335897 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-5c1bd9-6c468fdb4c-g7qfm"] Sep 11 12:16:01.338382 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:01.338364 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5c1bd9-6c468fdb4c-g7qfm" Sep 11 12:16:01.341313 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:01.341286 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-5c1bd9-dockercfg-h5hbv\"" Sep 11 12:16:01.341536 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:01.341518 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-5c1bd9\"" Sep 11 12:16:01.358898 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:01.358839 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-5c1bd9-6c468fdb4c-g7qfm"] Sep 11 12:16:01.384710 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:01.384677 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5c1bd9-57bd5c67cd-9knwh"] Sep 11 12:16:01.387220 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:01.387194 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5c1bd9-57bd5c67cd-9knwh" Sep 11 12:16:01.401544 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:01.401511 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5c1bd9-57bd5c67cd-9knwh"] Sep 11 12:16:01.523422 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:01.523381 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a74ee09-e4aa-4e74-a3db-b7aab95dca04-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-5c1bd9-6c468fdb4c-g7qfm\" (UID: \"2a74ee09-e4aa-4e74-a3db-b7aab95dca04\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5c1bd9-6c468fdb4c-g7qfm" Sep 11 12:16:01.523617 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:01.523442 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plrtg\" (UniqueName: \"kubernetes.io/projected/2a74ee09-e4aa-4e74-a3db-b7aab95dca04-kube-api-access-plrtg\") pod \"payload-processing-e2e-ipp-cleanup-5c1bd9-6c468fdb4c-g7qfm\" (UID: \"2a74ee09-e4aa-4e74-a3db-b7aab95dca04\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5c1bd9-6c468fdb4c-g7qfm" Sep 11 12:16:01.523617 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:01.523530 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcq8v\" (UniqueName: \"kubernetes.io/projected/ecc4a81b-a2ac-4661-9dfc-eda838871e02-kube-api-access-fcq8v\") pod \"payload-pre-processing-e2e-ipp-cleanup-5c1bd9-57bd5c67cd-9knwh\" (UID: \"ecc4a81b-a2ac-4661-9dfc-eda838871e02\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5c1bd9-57bd5c67cd-9knwh" Sep 11 12:16:01.523617 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:01.523590 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecc4a81b-a2ac-4661-9dfc-eda838871e02-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-5c1bd9-57bd5c67cd-9knwh\" (UID: \"ecc4a81b-a2ac-4661-9dfc-eda838871e02\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5c1bd9-57bd5c67cd-9knwh" Sep 11 12:16:01.624790 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:01.624692 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a74ee09-e4aa-4e74-a3db-b7aab95dca04-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-5c1bd9-6c468fdb4c-g7qfm\" (UID: \"2a74ee09-e4aa-4e74-a3db-b7aab95dca04\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5c1bd9-6c468fdb4c-g7qfm" Sep 11 12:16:01.624790 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:01.624745 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plrtg\" (UniqueName: \"kubernetes.io/projected/2a74ee09-e4aa-4e74-a3db-b7aab95dca04-kube-api-access-plrtg\") pod \"payload-processing-e2e-ipp-cleanup-5c1bd9-6c468fdb4c-g7qfm\" (UID: \"2a74ee09-e4aa-4e74-a3db-b7aab95dca04\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5c1bd9-6c468fdb4c-g7qfm" Sep 11 12:16:01.625081 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:01.624802 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcq8v\" (UniqueName: \"kubernetes.io/projected/ecc4a81b-a2ac-4661-9dfc-eda838871e02-kube-api-access-fcq8v\") pod \"payload-pre-processing-e2e-ipp-cleanup-5c1bd9-57bd5c67cd-9knwh\" (UID: \"ecc4a81b-a2ac-4661-9dfc-eda838871e02\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5c1bd9-57bd5c67cd-9knwh" Sep 11 12:16:01.625081 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:01.624852 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecc4a81b-a2ac-4661-9dfc-eda838871e02-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-5c1bd9-57bd5c67cd-9knwh\" (UID: \"ecc4a81b-a2ac-4661-9dfc-eda838871e02\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5c1bd9-57bd5c67cd-9knwh" Sep 11 12:16:01.625595 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:01.625574 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecc4a81b-a2ac-4661-9dfc-eda838871e02-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-5c1bd9-57bd5c67cd-9knwh\" (UID: \"ecc4a81b-a2ac-4661-9dfc-eda838871e02\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5c1bd9-57bd5c67cd-9knwh" Sep 11 12:16:01.625637 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:01.625584 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a74ee09-e4aa-4e74-a3db-b7aab95dca04-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-5c1bd9-6c468fdb4c-g7qfm\" (UID: \"2a74ee09-e4aa-4e74-a3db-b7aab95dca04\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5c1bd9-6c468fdb4c-g7qfm" Sep 11 12:16:01.633330 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:01.633299 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plrtg\" (UniqueName: \"kubernetes.io/projected/2a74ee09-e4aa-4e74-a3db-b7aab95dca04-kube-api-access-plrtg\") pod \"payload-processing-e2e-ipp-cleanup-5c1bd9-6c468fdb4c-g7qfm\" (UID: \"2a74ee09-e4aa-4e74-a3db-b7aab95dca04\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5c1bd9-6c468fdb4c-g7qfm" Sep 11 12:16:01.633470 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:01.633419 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcq8v\" (UniqueName: \"kubernetes.io/projected/ecc4a81b-a2ac-4661-9dfc-eda838871e02-kube-api-access-fcq8v\") pod \"payload-pre-processing-e2e-ipp-cleanup-5c1bd9-57bd5c67cd-9knwh\" (UID: \"ecc4a81b-a2ac-4661-9dfc-eda838871e02\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5c1bd9-57bd5c67cd-9knwh" Sep 11 12:16:01.649287 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:01.649263 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5c1bd9-6c468fdb4c-g7qfm" Sep 11 12:16:01.697472 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:01.697441 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5c1bd9-57bd5c67cd-9knwh" Sep 11 12:16:01.918830 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:01.918797 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-5c1bd9-6c468fdb4c-g7qfm"] Sep 11 12:16:01.920062 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:16:01.920035 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a74ee09_e4aa_4e74_a3db_b7aab95dca04.slice/crio-244bf295c8803fc2319aff784062f2b67c2be1572dd888297c959feda72d5301 WatchSource:0}: Error finding container 244bf295c8803fc2319aff784062f2b67c2be1572dd888297c959feda72d5301: Status 404 returned error can't find the container with id 244bf295c8803fc2319aff784062f2b67c2be1572dd888297c959feda72d5301 Sep 11 12:16:01.982532 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:01.982507 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5c1bd9-57bd5c67cd-9knwh"] Sep 11 12:16:01.985008 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:16:01.984986 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecc4a81b_a2ac_4661_9dfc_eda838871e02.slice/crio-f10ad9d91a38c4747801ed136851146a99bae6b96e212e1efb416119df040f37 WatchSource:0}: Error finding container f10ad9d91a38c4747801ed136851146a99bae6b96e212e1efb416119df040f37: Status 404 returned error can't find the container with id f10ad9d91a38c4747801ed136851146a99bae6b96e212e1efb416119df040f37 Sep 11 12:16:02.463283 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:02.463241 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5c1bd9-6c468fdb4c-g7qfm" event={"ID":"2a74ee09-e4aa-4e74-a3db-b7aab95dca04","Type":"ContainerStarted","Data":"857958d36e129ebd11e5a8bd759ea718cfbbf16f0f0c41bb7f7f97d75fd18c9b"} Sep 11 12:16:02.463775 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:02.463307 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5c1bd9-6c468fdb4c-g7qfm" event={"ID":"2a74ee09-e4aa-4e74-a3db-b7aab95dca04","Type":"ContainerStarted","Data":"244bf295c8803fc2319aff784062f2b67c2be1572dd888297c959feda72d5301"} Sep 11 12:16:02.463775 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:02.463458 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5c1bd9-6c468fdb4c-g7qfm" Sep 11 12:16:02.464993 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:02.464966 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5c1bd9-57bd5c67cd-9knwh" event={"ID":"ecc4a81b-a2ac-4661-9dfc-eda838871e02","Type":"ContainerStarted","Data":"1cca43e0dc5da2a214d6e7ae9a62ba653ca1b9baacf2cfb03170b95f5cb82772"} Sep 11 12:16:02.465121 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:02.465015 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5c1bd9-57bd5c67cd-9knwh" event={"ID":"ecc4a81b-a2ac-4661-9dfc-eda838871e02","Type":"ContainerStarted","Data":"f10ad9d91a38c4747801ed136851146a99bae6b96e212e1efb416119df040f37"} Sep 11 12:16:02.465121 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:02.465047 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5c1bd9-57bd5c67cd-9knwh" Sep 11 12:16:02.481454 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:02.481400 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5c1bd9-6c468fdb4c-g7qfm" podStartSLOduration=1.481386427 podStartE2EDuration="1.481386427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:16:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:16:02.478917375 +0000 UTC m=+1694.243890549" watchObservedRunningTime="2026-09-11 12:16:02.481386427 +0000 UTC m=+1694.246359599" Sep 11 12:16:02.498175 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:02.498113 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5c1bd9-57bd5c67cd-9knwh" podStartSLOduration=1.4981001489999999 podStartE2EDuration="1.498100149s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:16:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:16:02.496592809 +0000 UTC m=+1694.261566000" watchObservedRunningTime="2026-09-11 12:16:02.498100149 +0000 UTC m=+1694.263073325" Sep 11 12:16:13.471459 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:13.471430 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5c1bd9-57bd5c67cd-9knwh" Sep 11 12:16:13.471858 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:13.471579 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5c1bd9-6c468fdb4c-g7qfm" Sep 11 12:16:20.574054 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:20.574021 2566 generic.go:356] "Generic (PLEG): container finished" podID="33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0" containerID="019abb4bd9402f98aaa526f0c96671d91d557752757b7c9b87743364a593f84e" exitCode=0 Sep 11 12:16:20.574474 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:20.574093 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" event={"ID":"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0","Type":"ContainerDied","Data":"019abb4bd9402f98aaa526f0c96671d91d557752757b7c9b87743364a593f84e"} Sep 11 12:16:20.574518 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:20.574499 2566 scope.go:117] "RemoveContainer" containerID="019abb4bd9402f98aaa526f0c96671d91d557752757b7c9b87743364a593f84e" Sep 11 12:16:20.575644 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:20.575608 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:16:20.575746 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:20.575702 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:16:20.575813 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:20.575753 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 12:16:21.581398 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:21.581358 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" event={"ID":"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0","Type":"ContainerStarted","Data":"62759a30fb9789ffccd2771cd6f9b200c6c080a71d3aaf04d6d9caf73d1927b7"} Sep 11 12:16:22.035018 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:22.034915 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" Sep 11 12:16:22.035223 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:22.035199 2566 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Sep 11 12:16:22.035301 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:22.035252 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" podUID="33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Sep 11 12:16:23.034522 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:23.034481 2566 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Sep 11 12:16:23.034972 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:23.034549 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" podUID="33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Sep 11 12:16:24.034496 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:24.034454 2566 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Sep 11 12:16:24.034679 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:24.034514 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" podUID="33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Sep 11 12:16:25.034582 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:25.034542 2566 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Sep 11 12:16:25.034782 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:25.034602 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" podUID="33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Sep 11 12:16:25.080626 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:25.080584 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5c1bd9-zd7vs"] Sep 11 12:16:25.083347 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:25.083323 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5c1bd9-zd7vs" Sep 11 12:16:25.086517 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:25.086495 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9xkpf\"" Sep 11 12:16:25.094858 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:25.094833 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5c1bd9-zd7vs"] Sep 11 12:16:25.150255 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:25.150216 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44fb82a8-a515-4ff1-9684-b2eefcf50cb1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-5c1bd9-zd7vs\" (UID: \"44fb82a8-a515-4ff1-9684-b2eefcf50cb1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5c1bd9-zd7vs" Sep 11 12:16:25.251413 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:25.251375 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44fb82a8-a515-4ff1-9684-b2eefcf50cb1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-5c1bd9-zd7vs\" (UID: \"44fb82a8-a515-4ff1-9684-b2eefcf50cb1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5c1bd9-zd7vs" Sep 11 12:16:25.252055 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:25.252036 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44fb82a8-a515-4ff1-9684-b2eefcf50cb1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-5c1bd9-zd7vs\" (UID: \"44fb82a8-a515-4ff1-9684-b2eefcf50cb1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5c1bd9-zd7vs" Sep 11 12:16:25.394422 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:25.394364 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5c1bd9-zd7vs" Sep 11 12:16:25.683200 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:25.683170 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5c1bd9-zd7vs"] Sep 11 12:16:25.685469 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:16:25.685440 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44fb82a8_a515_4ff1_9684_b2eefcf50cb1.slice/crio-4e4e2d972653570c968188b9f920f1cebee09df8b86d1fedfb13259e34714a5d WatchSource:0}: Error finding container 4e4e2d972653570c968188b9f920f1cebee09df8b86d1fedfb13259e34714a5d: Status 404 returned error can't find the container with id 4e4e2d972653570c968188b9f920f1cebee09df8b86d1fedfb13259e34714a5d Sep 11 12:16:26.034601 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:26.034571 2566 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Sep 11 12:16:26.034721 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:26.034621 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" podUID="33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Sep 11 12:16:26.626643 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:26.626605 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5c1bd9-zd7vs" event={"ID":"44fb82a8-a515-4ff1-9684-b2eefcf50cb1","Type":"ContainerStarted","Data":"b3a0f161f37fa88d79d9b12c541a7a6926d4f12063086dc9659acaa8cca0ca81"} Sep 11 12:16:26.627191 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:26.626704 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5c1bd9-zd7vs" event={"ID":"44fb82a8-a515-4ff1-9684-b2eefcf50cb1","Type":"ContainerStarted","Data":"4e4e2d972653570c968188b9f920f1cebee09df8b86d1fedfb13259e34714a5d"} Sep 11 12:16:26.644287 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:26.644210 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5c1bd9-zd7vs" podStartSLOduration=1.644195447 podStartE2EDuration="1.644195447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:16:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:16:26.643414894 +0000 UTC m=+1718.408388071" watchObservedRunningTime="2026-09-11 12:16:26.644195447 +0000 UTC m=+1718.409168621" Sep 11 12:16:27.034433 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:27.034346 2566 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Sep 11 12:16:27.034433 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:27.034404 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" podUID="33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Sep 11 12:16:27.632852 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:27.632815 2566 generic.go:356] "Generic (PLEG): container finished" podID="44fb82a8-a515-4ff1-9684-b2eefcf50cb1" containerID="b3a0f161f37fa88d79d9b12c541a7a6926d4f12063086dc9659acaa8cca0ca81" exitCode=0 Sep 11 12:16:27.633253 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:27.632901 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5c1bd9-zd7vs" event={"ID":"44fb82a8-a515-4ff1-9684-b2eefcf50cb1","Type":"ContainerDied","Data":"b3a0f161f37fa88d79d9b12c541a7a6926d4f12063086dc9659acaa8cca0ca81"} Sep 11 12:16:28.038859 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:28.038783 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" Sep 11 12:16:28.039109 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:28.039085 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" Sep 11 12:16:28.039775 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:28.039755 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" Sep 11 12:16:28.764193 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:28.764169 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5c1bd9-zd7vs" Sep 11 12:16:28.885625 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:28.885587 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44fb82a8-a515-4ff1-9684-b2eefcf50cb1-maas-api-service-ca\") pod \"44fb82a8-a515-4ff1-9684-b2eefcf50cb1\" (UID: \"44fb82a8-a515-4ff1-9684-b2eefcf50cb1\") " Sep 11 12:16:28.886038 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:28.886012 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44fb82a8-a515-4ff1-9684-b2eefcf50cb1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "44fb82a8-a515-4ff1-9684-b2eefcf50cb1" (UID: "44fb82a8-a515-4ff1-9684-b2eefcf50cb1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 12:16:28.987281 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:28.987198 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44fb82a8-a515-4ff1-9684-b2eefcf50cb1-maas-api-service-ca\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:16:29.643938 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:29.643906 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5c1bd9-zd7vs" Sep 11 12:16:29.643938 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:29.643924 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5c1bd9-zd7vs" event={"ID":"44fb82a8-a515-4ff1-9684-b2eefcf50cb1","Type":"ContainerDied","Data":"4e4e2d972653570c968188b9f920f1cebee09df8b86d1fedfb13259e34714a5d"} Sep 11 12:16:29.644239 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:29.643981 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4e4e2d972653570c968188b9f920f1cebee09df8b86d1fedfb13259e34714a5d" Sep 11 12:16:31.270385 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:31.270350 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-5c1bd9-6c468fdb4c-g7qfm"] Sep 11 12:16:31.270810 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:31.270595 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5c1bd9-6c468fdb4c-g7qfm" podUID="2a74ee09-e4aa-4e74-a3db-b7aab95dca04" containerName="payload-processing" containerID="cri-o://857958d36e129ebd11e5a8bd759ea718cfbbf16f0f0c41bb7f7f97d75fd18c9b" gracePeriod=30 Sep 11 12:16:31.275301 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:31.275271 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5c1bd9-57bd5c67cd-9knwh"] Sep 11 12:16:31.275668 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:31.275507 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5c1bd9-57bd5c67cd-9knwh" podUID="ecc4a81b-a2ac-4661-9dfc-eda838871e02" containerName="payload-pre-processing" containerID="cri-o://1cca43e0dc5da2a214d6e7ae9a62ba653ca1b9baacf2cfb03170b95f5cb82772" gracePeriod=30 Sep 11 12:16:31.506888 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:31.504204 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5c1bd9-zd7vs"] Sep 11 12:16:31.507281 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:31.507240 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5c1bd9-zd7vs"] Sep 11 12:16:31.548879 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:31.548843 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5c1bd9-6c468fdb4c-g7qfm" Sep 11 12:16:31.580489 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:31.580464 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5c1bd9-57bd5c67cd-9knwh" Sep 11 12:16:31.614315 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:31.614277 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-plrtg\" (UniqueName: \"kubernetes.io/projected/2a74ee09-e4aa-4e74-a3db-b7aab95dca04-kube-api-access-plrtg\") pod \"2a74ee09-e4aa-4e74-a3db-b7aab95dca04\" (UID: \"2a74ee09-e4aa-4e74-a3db-b7aab95dca04\") " Sep 11 12:16:31.614496 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:31.614384 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a74ee09-e4aa-4e74-a3db-b7aab95dca04-plugins-config-volume\") pod \"2a74ee09-e4aa-4e74-a3db-b7aab95dca04\" (UID: \"2a74ee09-e4aa-4e74-a3db-b7aab95dca04\") " Sep 11 12:16:31.614907 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:31.614859 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a74ee09-e4aa-4e74-a3db-b7aab95dca04-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2a74ee09-e4aa-4e74-a3db-b7aab95dca04" (UID: "2a74ee09-e4aa-4e74-a3db-b7aab95dca04"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 12:16:31.616446 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:31.616418 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a74ee09-e4aa-4e74-a3db-b7aab95dca04-kube-api-access-plrtg" (OuterVolumeSpecName: "kube-api-access-plrtg") pod "2a74ee09-e4aa-4e74-a3db-b7aab95dca04" (UID: "2a74ee09-e4aa-4e74-a3db-b7aab95dca04"). InnerVolumeSpecName "kube-api-access-plrtg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:16:31.654434 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:31.654389 2566 generic.go:356] "Generic (PLEG): container finished" podID="2a74ee09-e4aa-4e74-a3db-b7aab95dca04" containerID="857958d36e129ebd11e5a8bd759ea718cfbbf16f0f0c41bb7f7f97d75fd18c9b" exitCode=0 Sep 11 12:16:31.654609 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:31.654442 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5c1bd9-6c468fdb4c-g7qfm" Sep 11 12:16:31.654609 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:31.654485 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5c1bd9-6c468fdb4c-g7qfm" event={"ID":"2a74ee09-e4aa-4e74-a3db-b7aab95dca04","Type":"ContainerDied","Data":"857958d36e129ebd11e5a8bd759ea718cfbbf16f0f0c41bb7f7f97d75fd18c9b"} Sep 11 12:16:31.654609 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:31.654562 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5c1bd9-6c468fdb4c-g7qfm" event={"ID":"2a74ee09-e4aa-4e74-a3db-b7aab95dca04","Type":"ContainerDied","Data":"244bf295c8803fc2319aff784062f2b67c2be1572dd888297c959feda72d5301"} Sep 11 12:16:31.654609 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:31.654604 2566 scope.go:117] "RemoveContainer" containerID="857958d36e129ebd11e5a8bd759ea718cfbbf16f0f0c41bb7f7f97d75fd18c9b" Sep 11 12:16:31.655998 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:31.655977 2566 generic.go:356] "Generic (PLEG): container finished" podID="ecc4a81b-a2ac-4661-9dfc-eda838871e02" containerID="1cca43e0dc5da2a214d6e7ae9a62ba653ca1b9baacf2cfb03170b95f5cb82772" exitCode=0 Sep 11 12:16:31.656127 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:31.656028 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5c1bd9-57bd5c67cd-9knwh" Sep 11 12:16:31.656127 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:31.656055 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5c1bd9-57bd5c67cd-9knwh" event={"ID":"ecc4a81b-a2ac-4661-9dfc-eda838871e02","Type":"ContainerDied","Data":"1cca43e0dc5da2a214d6e7ae9a62ba653ca1b9baacf2cfb03170b95f5cb82772"} Sep 11 12:16:31.656127 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:31.656116 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5c1bd9-57bd5c67cd-9knwh" event={"ID":"ecc4a81b-a2ac-4661-9dfc-eda838871e02","Type":"ContainerDied","Data":"f10ad9d91a38c4747801ed136851146a99bae6b96e212e1efb416119df040f37"} Sep 11 12:16:31.664172 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:31.664153 2566 scope.go:117] "RemoveContainer" containerID="857958d36e129ebd11e5a8bd759ea718cfbbf16f0f0c41bb7f7f97d75fd18c9b" Sep 11 12:16:31.664440 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:16:31.664419 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"857958d36e129ebd11e5a8bd759ea718cfbbf16f0f0c41bb7f7f97d75fd18c9b\": container with ID starting with 857958d36e129ebd11e5a8bd759ea718cfbbf16f0f0c41bb7f7f97d75fd18c9b not found: ID does not exist" containerID="857958d36e129ebd11e5a8bd759ea718cfbbf16f0f0c41bb7f7f97d75fd18c9b" Sep 11 12:16:31.664488 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:31.664450 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"857958d36e129ebd11e5a8bd759ea718cfbbf16f0f0c41bb7f7f97d75fd18c9b"} err="failed to get container status \"857958d36e129ebd11e5a8bd759ea718cfbbf16f0f0c41bb7f7f97d75fd18c9b\": rpc error: code = NotFound desc = could not find container \"857958d36e129ebd11e5a8bd759ea718cfbbf16f0f0c41bb7f7f97d75fd18c9b\": container with ID starting with 857958d36e129ebd11e5a8bd759ea718cfbbf16f0f0c41bb7f7f97d75fd18c9b not found: ID does not exist" Sep 11 12:16:31.664488 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:31.664467 2566 scope.go:117] "RemoveContainer" containerID="1cca43e0dc5da2a214d6e7ae9a62ba653ca1b9baacf2cfb03170b95f5cb82772" Sep 11 12:16:31.672521 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:31.672501 2566 scope.go:117] "RemoveContainer" containerID="1cca43e0dc5da2a214d6e7ae9a62ba653ca1b9baacf2cfb03170b95f5cb82772" Sep 11 12:16:31.672789 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:16:31.672771 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cca43e0dc5da2a214d6e7ae9a62ba653ca1b9baacf2cfb03170b95f5cb82772\": container with ID starting with 1cca43e0dc5da2a214d6e7ae9a62ba653ca1b9baacf2cfb03170b95f5cb82772 not found: ID does not exist" containerID="1cca43e0dc5da2a214d6e7ae9a62ba653ca1b9baacf2cfb03170b95f5cb82772" Sep 11 12:16:31.672835 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:31.672796 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cca43e0dc5da2a214d6e7ae9a62ba653ca1b9baacf2cfb03170b95f5cb82772"} err="failed to get container status \"1cca43e0dc5da2a214d6e7ae9a62ba653ca1b9baacf2cfb03170b95f5cb82772\": rpc error: code = NotFound desc = could not find container \"1cca43e0dc5da2a214d6e7ae9a62ba653ca1b9baacf2cfb03170b95f5cb82772\": container with ID starting with 1cca43e0dc5da2a214d6e7ae9a62ba653ca1b9baacf2cfb03170b95f5cb82772 not found: ID does not exist" Sep 11 12:16:31.676336 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:31.676314 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-5c1bd9-6c468fdb4c-g7qfm"] Sep 11 12:16:31.680429 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:31.680406 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-5c1bd9-6c468fdb4c-g7qfm"] Sep 11 12:16:31.715139 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:31.715114 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecc4a81b-a2ac-4661-9dfc-eda838871e02-plugins-config-volume\") pod \"ecc4a81b-a2ac-4661-9dfc-eda838871e02\" (UID: \"ecc4a81b-a2ac-4661-9dfc-eda838871e02\") " Sep 11 12:16:31.715231 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:31.715175 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fcq8v\" (UniqueName: \"kubernetes.io/projected/ecc4a81b-a2ac-4661-9dfc-eda838871e02-kube-api-access-fcq8v\") pod \"ecc4a81b-a2ac-4661-9dfc-eda838871e02\" (UID: \"ecc4a81b-a2ac-4661-9dfc-eda838871e02\") " Sep 11 12:16:31.715465 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:31.715437 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecc4a81b-a2ac-4661-9dfc-eda838871e02-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ecc4a81b-a2ac-4661-9dfc-eda838871e02" (UID: "ecc4a81b-a2ac-4661-9dfc-eda838871e02"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 12:16:31.715516 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:31.715483 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plrtg\" (UniqueName: \"kubernetes.io/projected/2a74ee09-e4aa-4e74-a3db-b7aab95dca04-kube-api-access-plrtg\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:16:31.715516 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:31.715499 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a74ee09-e4aa-4e74-a3db-b7aab95dca04-plugins-config-volume\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:16:31.717194 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:31.717169 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecc4a81b-a2ac-4661-9dfc-eda838871e02-kube-api-access-fcq8v" (OuterVolumeSpecName: "kube-api-access-fcq8v") pod "ecc4a81b-a2ac-4661-9dfc-eda838871e02" (UID: "ecc4a81b-a2ac-4661-9dfc-eda838871e02"). InnerVolumeSpecName "kube-api-access-fcq8v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:16:31.816826 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:31.816796 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecc4a81b-a2ac-4661-9dfc-eda838871e02-plugins-config-volume\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:16:31.816826 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:31.816826 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fcq8v\" (UniqueName: \"kubernetes.io/projected/ecc4a81b-a2ac-4661-9dfc-eda838871e02-kube-api-access-fcq8v\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:16:31.977521 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:31.977492 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5c1bd9-57bd5c67cd-9knwh"] Sep 11 12:16:31.983393 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:31.983367 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5c1bd9-57bd5c67cd-9knwh"] Sep 11 12:16:32.758197 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:32.758164 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a74ee09-e4aa-4e74-a3db-b7aab95dca04" path="/var/lib/kubelet/pods/2a74ee09-e4aa-4e74-a3db-b7aab95dca04/volumes" Sep 11 12:16:32.758563 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:32.758491 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44fb82a8-a515-4ff1-9684-b2eefcf50cb1" path="/var/lib/kubelet/pods/44fb82a8-a515-4ff1-9684-b2eefcf50cb1/volumes" Sep 11 12:16:32.758804 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:32.758791 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecc4a81b-a2ac-4661-9dfc-eda838871e02" path="/var/lib/kubelet/pods/ecc4a81b-a2ac-4661-9dfc-eda838871e02/volumes" Sep 11 12:16:38.180888 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:38.180844 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq"] Sep 11 12:16:38.181291 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:38.181064 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" podUID="33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0" containerName="istio-proxy" containerID="cri-o://62759a30fb9789ffccd2771cd6f9b200c6c080a71d3aaf04d6d9caf73d1927b7" gracePeriod=30 Sep 11 12:16:43.041414 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.041379 2566 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 11 12:16:43.041840 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.041445 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" podUID="33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 12:16:43.440495 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.440471 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" Sep 11 12:16:43.531640 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.531612 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-istio-data\") pod \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\" (UID: \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\") " Sep 11 12:16:43.531833 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.531655 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-workload-socket\") pod \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\" (UID: \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\") " Sep 11 12:16:43.531833 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.531676 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-credential-socket\") pod \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\" (UID: \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\") " Sep 11 12:16:43.531833 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.531697 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mz5d6\" (UniqueName: \"kubernetes.io/projected/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-kube-api-access-mz5d6\") pod \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\" (UID: \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\") " Sep 11 12:16:43.531833 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.531712 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-istio-token\") pod \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\" (UID: \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\") " Sep 11 12:16:43.531833 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.531771 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-istio-envoy\") pod \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\" (UID: \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\") " Sep 11 12:16:43.531833 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.531795 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-istiod-ca-cert\") pod \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\" (UID: \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\") " Sep 11 12:16:43.532183 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.531891 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-workload-certs\") pod \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\" (UID: \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\") " Sep 11 12:16:43.532183 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.531928 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-istio-podinfo\") pod \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\" (UID: \"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0\") " Sep 11 12:16:43.532183 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.531963 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-istio-data" (OuterVolumeSpecName: "istio-data") pod "33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0" (UID: "33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:16:43.532183 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.531971 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0" (UID: "33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:16:43.532183 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.532050 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0" (UID: "33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:16:43.532183 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.532159 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0" (UID: "33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:16:43.532465 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.532259 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0" (UID: "33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 12:16:43.532465 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.532271 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-istio-data\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:16:43.532465 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.532330 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-workload-socket\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:16:43.532465 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.532347 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-credential-socket\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:16:43.532465 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.532364 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-workload-certs\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:16:43.534019 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.533995 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0" (UID: "33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:16:43.534299 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.534276 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-istio-token" (OuterVolumeSpecName: "istio-token") pod "33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0" (UID: "33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:16:43.534475 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.534457 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0" (UID: "33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 11 12:16:43.534532 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.534465 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-kube-api-access-mz5d6" (OuterVolumeSpecName: "kube-api-access-mz5d6") pod "33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0" (UID: "33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0"). InnerVolumeSpecName "kube-api-access-mz5d6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:16:43.633238 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.633205 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-istio-token\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:16:43.633238 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.633233 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-istio-envoy\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:16:43.633450 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.633247 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-istiod-ca-cert\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:16:43.633450 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.633258 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-istio-podinfo\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:16:43.633450 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.633269 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mz5d6\" (UniqueName: \"kubernetes.io/projected/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0-kube-api-access-mz5d6\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:16:43.721998 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.721967 2566 generic.go:356] "Generic (PLEG): container finished" podID="33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0" containerID="62759a30fb9789ffccd2771cd6f9b200c6c080a71d3aaf04d6d9caf73d1927b7" exitCode=0 Sep 11 12:16:43.722140 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.722015 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" Sep 11 12:16:43.722140 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.722016 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" event={"ID":"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0","Type":"ContainerDied","Data":"62759a30fb9789ffccd2771cd6f9b200c6c080a71d3aaf04d6d9caf73d1927b7"} Sep 11 12:16:43.722217 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.722141 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq" event={"ID":"33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0","Type":"ContainerDied","Data":"7ef42982b12e30527c92e1560cd342d06b8d1f94cc2275a623acd90091944515"} Sep 11 12:16:43.722217 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.722171 2566 scope.go:117] "RemoveContainer" containerID="62759a30fb9789ffccd2771cd6f9b200c6c080a71d3aaf04d6d9caf73d1927b7" Sep 11 12:16:43.732364 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.732344 2566 scope.go:117] "RemoveContainer" containerID="019abb4bd9402f98aaa526f0c96671d91d557752757b7c9b87743364a593f84e" Sep 11 12:16:43.739641 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.739625 2566 scope.go:117] "RemoveContainer" containerID="62759a30fb9789ffccd2771cd6f9b200c6c080a71d3aaf04d6d9caf73d1927b7" Sep 11 12:16:43.739913 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:16:43.739893 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62759a30fb9789ffccd2771cd6f9b200c6c080a71d3aaf04d6d9caf73d1927b7\": container with ID starting with 62759a30fb9789ffccd2771cd6f9b200c6c080a71d3aaf04d6d9caf73d1927b7 not found: ID does not exist" containerID="62759a30fb9789ffccd2771cd6f9b200c6c080a71d3aaf04d6d9caf73d1927b7" Sep 11 12:16:43.739972 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.739920 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62759a30fb9789ffccd2771cd6f9b200c6c080a71d3aaf04d6d9caf73d1927b7"} err="failed to get container status \"62759a30fb9789ffccd2771cd6f9b200c6c080a71d3aaf04d6d9caf73d1927b7\": rpc error: code = NotFound desc = could not find container \"62759a30fb9789ffccd2771cd6f9b200c6c080a71d3aaf04d6d9caf73d1927b7\": container with ID starting with 62759a30fb9789ffccd2771cd6f9b200c6c080a71d3aaf04d6d9caf73d1927b7 not found: ID does not exist" Sep 11 12:16:43.739972 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.739938 2566 scope.go:117] "RemoveContainer" containerID="019abb4bd9402f98aaa526f0c96671d91d557752757b7c9b87743364a593f84e" Sep 11 12:16:43.740179 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:16:43.740163 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"019abb4bd9402f98aaa526f0c96671d91d557752757b7c9b87743364a593f84e\": container with ID starting with 019abb4bd9402f98aaa526f0c96671d91d557752757b7c9b87743364a593f84e not found: ID does not exist" containerID="019abb4bd9402f98aaa526f0c96671d91d557752757b7c9b87743364a593f84e" Sep 11 12:16:43.740221 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.740184 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"019abb4bd9402f98aaa526f0c96671d91d557752757b7c9b87743364a593f84e"} err="failed to get container status \"019abb4bd9402f98aaa526f0c96671d91d557752757b7c9b87743364a593f84e\": rpc error: code = NotFound desc = could not find container \"019abb4bd9402f98aaa526f0c96671d91d557752757b7c9b87743364a593f84e\": container with ID starting with 019abb4bd9402f98aaa526f0c96671d91d557752757b7c9b87743364a593f84e not found: ID does not exist" Sep 11 12:16:43.764619 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.764590 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq"] Sep 11 12:16:43.772215 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:43.772197 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-5c1bd9-openshift-default-f76fc9c56-f9wgq"] Sep 11 12:16:44.758023 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:44.757990 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0" path="/var/lib/kubelet/pods/33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0/volumes" Sep 11 12:16:57.411942 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:57.411904 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-bf9d5c-7c46789f87-94c7j"] Sep 11 12:16:57.412423 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:57.412145 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-bf9d5c-7c46789f87-94c7j" podUID="74844184-6bfc-476f-b940-14bbb7018f24" containerName="payload-processing" containerID="cri-o://2f05796f30bb3681f9365f6bc7d6a5d7a3f8177a14b12f71e8c81896b932d7aa" gracePeriod=30 Sep 11 12:16:57.656356 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:57.656328 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-bf9d5c-7c46789f87-94c7j" Sep 11 12:16:57.756701 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:57.756569 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74844184-6bfc-476f-b940-14bbb7018f24-plugins-config-volume\") pod \"74844184-6bfc-476f-b940-14bbb7018f24\" (UID: \"74844184-6bfc-476f-b940-14bbb7018f24\") " Sep 11 12:16:57.756701 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:57.756656 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7ngfd\" (UniqueName: \"kubernetes.io/projected/74844184-6bfc-476f-b940-14bbb7018f24-kube-api-access-7ngfd\") pod \"74844184-6bfc-476f-b940-14bbb7018f24\" (UID: \"74844184-6bfc-476f-b940-14bbb7018f24\") " Sep 11 12:16:57.757226 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:57.757196 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74844184-6bfc-476f-b940-14bbb7018f24-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "74844184-6bfc-476f-b940-14bbb7018f24" (UID: "74844184-6bfc-476f-b940-14bbb7018f24"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 12:16:57.759624 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:57.759592 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74844184-6bfc-476f-b940-14bbb7018f24-kube-api-access-7ngfd" (OuterVolumeSpecName: "kube-api-access-7ngfd") pod "74844184-6bfc-476f-b940-14bbb7018f24" (UID: "74844184-6bfc-476f-b940-14bbb7018f24"). InnerVolumeSpecName "kube-api-access-7ngfd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:16:57.792361 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:57.792326 2566 generic.go:356] "Generic (PLEG): container finished" podID="74844184-6bfc-476f-b940-14bbb7018f24" containerID="2f05796f30bb3681f9365f6bc7d6a5d7a3f8177a14b12f71e8c81896b932d7aa" exitCode=0 Sep 11 12:16:57.792497 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:57.792374 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-bf9d5c-7c46789f87-94c7j" Sep 11 12:16:57.792497 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:57.792373 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-bf9d5c-7c46789f87-94c7j" event={"ID":"74844184-6bfc-476f-b940-14bbb7018f24","Type":"ContainerDied","Data":"2f05796f30bb3681f9365f6bc7d6a5d7a3f8177a14b12f71e8c81896b932d7aa"} Sep 11 12:16:57.792599 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:57.792506 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-bf9d5c-7c46789f87-94c7j" event={"ID":"74844184-6bfc-476f-b940-14bbb7018f24","Type":"ContainerDied","Data":"31efd6346aef59a74b049f22349b12a4bb1679cc34eaf61b95e9657825de53d9"} Sep 11 12:16:57.792599 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:57.792535 2566 scope.go:117] "RemoveContainer" containerID="2f05796f30bb3681f9365f6bc7d6a5d7a3f8177a14b12f71e8c81896b932d7aa" Sep 11 12:16:57.801526 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:57.801506 2566 scope.go:117] "RemoveContainer" containerID="2f05796f30bb3681f9365f6bc7d6a5d7a3f8177a14b12f71e8c81896b932d7aa" Sep 11 12:16:57.801793 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:16:57.801777 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f05796f30bb3681f9365f6bc7d6a5d7a3f8177a14b12f71e8c81896b932d7aa\": container with ID starting with 2f05796f30bb3681f9365f6bc7d6a5d7a3f8177a14b12f71e8c81896b932d7aa not found: ID does not exist" containerID="2f05796f30bb3681f9365f6bc7d6a5d7a3f8177a14b12f71e8c81896b932d7aa" Sep 11 12:16:57.801842 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:57.801801 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f05796f30bb3681f9365f6bc7d6a5d7a3f8177a14b12f71e8c81896b932d7aa"} err="failed to get container status \"2f05796f30bb3681f9365f6bc7d6a5d7a3f8177a14b12f71e8c81896b932d7aa\": rpc error: code = NotFound desc = could not find container \"2f05796f30bb3681f9365f6bc7d6a5d7a3f8177a14b12f71e8c81896b932d7aa\": container with ID starting with 2f05796f30bb3681f9365f6bc7d6a5d7a3f8177a14b12f71e8c81896b932d7aa not found: ID does not exist" Sep 11 12:16:57.825736 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:57.825707 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-bf9d5c-7c46789f87-94c7j"] Sep 11 12:16:57.835748 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:57.835473 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-bf9d5c-7c46789f87-94c7j"] Sep 11 12:16:57.857277 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:57.857246 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74844184-6bfc-476f-b940-14bbb7018f24-plugins-config-volume\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:16:57.857277 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:57.857272 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ngfd\" (UniqueName: \"kubernetes.io/projected/74844184-6bfc-476f-b940-14bbb7018f24-kube-api-access-7ngfd\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:16:58.759166 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:16:58.759125 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74844184-6bfc-476f-b940-14bbb7018f24" path="/var/lib/kubelet/pods/74844184-6bfc-476f-b940-14bbb7018f24/volumes" Sep 11 12:18:19.490735 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:19.490702 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7"] Sep 11 12:18:19.491299 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:19.491027 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" podUID="b6dd23c7-c2ad-4620-a40e-de06161ee314" containerName="istio-proxy" containerID="cri-o://30705e049347ed93c08caa39aeb3f83e19a0cc8dd3ca3241a5c130c4f03f7e05" gracePeriod=30 Sep 11 12:18:24.743988 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:24.743962 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" Sep 11 12:18:24.800663 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:24.800561 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6dd23c7-c2ad-4620-a40e-de06161ee314-istio-data\") pod \"b6dd23c7-c2ad-4620-a40e-de06161ee314\" (UID: \"b6dd23c7-c2ad-4620-a40e-de06161ee314\") " Sep 11 12:18:24.800663 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:24.800625 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6dd23c7-c2ad-4620-a40e-de06161ee314-istio-token\") pod \"b6dd23c7-c2ad-4620-a40e-de06161ee314\" (UID: \"b6dd23c7-c2ad-4620-a40e-de06161ee314\") " Sep 11 12:18:24.800663 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:24.800644 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6dd23c7-c2ad-4620-a40e-de06161ee314-workload-certs\") pod \"b6dd23c7-c2ad-4620-a40e-de06161ee314\" (UID: \"b6dd23c7-c2ad-4620-a40e-de06161ee314\") " Sep 11 12:18:24.800663 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:24.800662 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6dd23c7-c2ad-4620-a40e-de06161ee314-istio-podinfo\") pod \"b6dd23c7-c2ad-4620-a40e-de06161ee314\" (UID: \"b6dd23c7-c2ad-4620-a40e-de06161ee314\") " Sep 11 12:18:24.801007 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:24.800682 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6dd23c7-c2ad-4620-a40e-de06161ee314-credential-socket\") pod \"b6dd23c7-c2ad-4620-a40e-de06161ee314\" (UID: \"b6dd23c7-c2ad-4620-a40e-de06161ee314\") " Sep 11 12:18:24.801007 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:24.800709 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6dd23c7-c2ad-4620-a40e-de06161ee314-workload-socket\") pod \"b6dd23c7-c2ad-4620-a40e-de06161ee314\" (UID: \"b6dd23c7-c2ad-4620-a40e-de06161ee314\") " Sep 11 12:18:24.801007 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:24.800728 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6dd23c7-c2ad-4620-a40e-de06161ee314-istio-envoy\") pod \"b6dd23c7-c2ad-4620-a40e-de06161ee314\" (UID: \"b6dd23c7-c2ad-4620-a40e-de06161ee314\") " Sep 11 12:18:24.801007 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:24.800825 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6dd23c7-c2ad-4620-a40e-de06161ee314-istiod-ca-cert\") pod \"b6dd23c7-c2ad-4620-a40e-de06161ee314\" (UID: \"b6dd23c7-c2ad-4620-a40e-de06161ee314\") " Sep 11 12:18:24.801007 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:24.800850 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-68kq9\" (UniqueName: \"kubernetes.io/projected/b6dd23c7-c2ad-4620-a40e-de06161ee314-kube-api-access-68kq9\") pod \"b6dd23c7-c2ad-4620-a40e-de06161ee314\" (UID: \"b6dd23c7-c2ad-4620-a40e-de06161ee314\") " Sep 11 12:18:24.801007 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:24.800840 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6dd23c7-c2ad-4620-a40e-de06161ee314-istio-data" (OuterVolumeSpecName: "istio-data") pod "b6dd23c7-c2ad-4620-a40e-de06161ee314" (UID: "b6dd23c7-c2ad-4620-a40e-de06161ee314"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:18:24.801408 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:24.801375 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6dd23c7-c2ad-4620-a40e-de06161ee314-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b6dd23c7-c2ad-4620-a40e-de06161ee314" (UID: "b6dd23c7-c2ad-4620-a40e-de06161ee314"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:18:24.801408 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:24.801395 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6dd23c7-c2ad-4620-a40e-de06161ee314-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b6dd23c7-c2ad-4620-a40e-de06161ee314" (UID: "b6dd23c7-c2ad-4620-a40e-de06161ee314"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:18:24.801537 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:24.801492 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6dd23c7-c2ad-4620-a40e-de06161ee314-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b6dd23c7-c2ad-4620-a40e-de06161ee314" (UID: "b6dd23c7-c2ad-4620-a40e-de06161ee314"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:18:24.801861 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:24.801786 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6dd23c7-c2ad-4620-a40e-de06161ee314-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b6dd23c7-c2ad-4620-a40e-de06161ee314" (UID: "b6dd23c7-c2ad-4620-a40e-de06161ee314"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 12:18:24.804950 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:24.802576 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6dd23c7-c2ad-4620-a40e-de06161ee314-workload-socket\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:18:24.804950 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:24.802606 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6dd23c7-c2ad-4620-a40e-de06161ee314-istiod-ca-cert\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:18:24.804950 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:24.802624 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6dd23c7-c2ad-4620-a40e-de06161ee314-istio-data\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:18:24.804950 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:24.802646 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6dd23c7-c2ad-4620-a40e-de06161ee314-workload-certs\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:18:24.804950 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:24.802661 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6dd23c7-c2ad-4620-a40e-de06161ee314-credential-socket\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:18:24.804950 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:24.803765 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6dd23c7-c2ad-4620-a40e-de06161ee314-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b6dd23c7-c2ad-4620-a40e-de06161ee314" (UID: "b6dd23c7-c2ad-4620-a40e-de06161ee314"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:18:24.804950 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:24.803778 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6dd23c7-c2ad-4620-a40e-de06161ee314-istio-token" (OuterVolumeSpecName: "istio-token") pod "b6dd23c7-c2ad-4620-a40e-de06161ee314" (UID: "b6dd23c7-c2ad-4620-a40e-de06161ee314"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:18:24.804950 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:24.804049 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b6dd23c7-c2ad-4620-a40e-de06161ee314-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b6dd23c7-c2ad-4620-a40e-de06161ee314" (UID: "b6dd23c7-c2ad-4620-a40e-de06161ee314"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 11 12:18:24.809543 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:24.809515 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6dd23c7-c2ad-4620-a40e-de06161ee314-kube-api-access-68kq9" (OuterVolumeSpecName: "kube-api-access-68kq9") pod "b6dd23c7-c2ad-4620-a40e-de06161ee314" (UID: "b6dd23c7-c2ad-4620-a40e-de06161ee314"). InnerVolumeSpecName "kube-api-access-68kq9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:18:24.903817 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:24.903775 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68kq9\" (UniqueName: \"kubernetes.io/projected/b6dd23c7-c2ad-4620-a40e-de06161ee314-kube-api-access-68kq9\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:18:24.903817 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:24.903813 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6dd23c7-c2ad-4620-a40e-de06161ee314-istio-token\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:18:24.903817 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:24.903823 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6dd23c7-c2ad-4620-a40e-de06161ee314-istio-podinfo\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:18:24.904094 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:24.903832 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6dd23c7-c2ad-4620-a40e-de06161ee314-istio-envoy\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:18:25.218851 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:25.218815 2566 generic.go:356] "Generic (PLEG): container finished" podID="b6dd23c7-c2ad-4620-a40e-de06161ee314" containerID="30705e049347ed93c08caa39aeb3f83e19a0cc8dd3ca3241a5c130c4f03f7e05" exitCode=0 Sep 11 12:18:25.219072 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:25.218897 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" Sep 11 12:18:25.219072 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:25.218905 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" event={"ID":"b6dd23c7-c2ad-4620-a40e-de06161ee314","Type":"ContainerDied","Data":"30705e049347ed93c08caa39aeb3f83e19a0cc8dd3ca3241a5c130c4f03f7e05"} Sep 11 12:18:25.219072 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:25.218989 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7" event={"ID":"b6dd23c7-c2ad-4620-a40e-de06161ee314","Type":"ContainerDied","Data":"dea80391550c7853b1298ac81532c39be906e1b163148f436ce663af6298e86b"} Sep 11 12:18:25.219072 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:25.219011 2566 scope.go:117] "RemoveContainer" containerID="30705e049347ed93c08caa39aeb3f83e19a0cc8dd3ca3241a5c130c4f03f7e05" Sep 11 12:18:25.227728 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:25.227708 2566 scope.go:117] "RemoveContainer" containerID="b6696c33cfec5fd5b4a3513e0a74e94fc70a2639f38deb7001d9ccdb3820805e" Sep 11 12:18:25.235680 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:25.235645 2566 scope.go:117] "RemoveContainer" containerID="30705e049347ed93c08caa39aeb3f83e19a0cc8dd3ca3241a5c130c4f03f7e05" Sep 11 12:18:25.235963 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:18:25.235946 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30705e049347ed93c08caa39aeb3f83e19a0cc8dd3ca3241a5c130c4f03f7e05\": container with ID starting with 30705e049347ed93c08caa39aeb3f83e19a0cc8dd3ca3241a5c130c4f03f7e05 not found: ID does not exist" containerID="30705e049347ed93c08caa39aeb3f83e19a0cc8dd3ca3241a5c130c4f03f7e05" Sep 11 12:18:25.236016 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:25.235972 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30705e049347ed93c08caa39aeb3f83e19a0cc8dd3ca3241a5c130c4f03f7e05"} err="failed to get container status \"30705e049347ed93c08caa39aeb3f83e19a0cc8dd3ca3241a5c130c4f03f7e05\": rpc error: code = NotFound desc = could not find container \"30705e049347ed93c08caa39aeb3f83e19a0cc8dd3ca3241a5c130c4f03f7e05\": container with ID starting with 30705e049347ed93c08caa39aeb3f83e19a0cc8dd3ca3241a5c130c4f03f7e05 not found: ID does not exist" Sep 11 12:18:25.236016 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:25.235990 2566 scope.go:117] "RemoveContainer" containerID="b6696c33cfec5fd5b4a3513e0a74e94fc70a2639f38deb7001d9ccdb3820805e" Sep 11 12:18:25.236248 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:18:25.236235 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6696c33cfec5fd5b4a3513e0a74e94fc70a2639f38deb7001d9ccdb3820805e\": container with ID starting with b6696c33cfec5fd5b4a3513e0a74e94fc70a2639f38deb7001d9ccdb3820805e not found: ID does not exist" containerID="b6696c33cfec5fd5b4a3513e0a74e94fc70a2639f38deb7001d9ccdb3820805e" Sep 11 12:18:25.236307 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:25.236250 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6696c33cfec5fd5b4a3513e0a74e94fc70a2639f38deb7001d9ccdb3820805e"} err="failed to get container status \"b6696c33cfec5fd5b4a3513e0a74e94fc70a2639f38deb7001d9ccdb3820805e\": rpc error: code = NotFound desc = could not find container \"b6696c33cfec5fd5b4a3513e0a74e94fc70a2639f38deb7001d9ccdb3820805e\": container with ID starting with b6696c33cfec5fd5b4a3513e0a74e94fc70a2639f38deb7001d9ccdb3820805e not found: ID does not exist" Sep 11 12:18:25.247379 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:25.247334 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7"] Sep 11 12:18:25.249777 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:25.249753 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-986cf2-openshift-default-77c6cd5f8d-nstr7"] Sep 11 12:18:26.758486 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:26.758449 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6dd23c7-c2ad-4620-a40e-de06161ee314" path="/var/lib/kubelet/pods/b6dd23c7-c2ad-4620-a40e-de06161ee314/volumes" Sep 11 12:18:30.637583 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:30.637551 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-af081c-29818815-bfgnh"] Sep 11 12:18:30.640911 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:30.640880 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-af081c-29818815-bfgnh"] Sep 11 12:18:30.668266 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:30.668219 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-af081c-74b8976d5c-tvm6b"] Sep 11 12:18:30.668822 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:30.668783 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-af081c-74b8976d5c-tvm6b" podUID="96a0d2ef-39dd-476e-a527-53e9af9cca98" containerName="payload-processing" containerID="cri-o://f9e69622b81d1cd118d4e0bf38c64eecf6368e78f0228a9bbf3259522838d61a" gracePeriod=30 Sep 11 12:18:30.758590 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:30.758551 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8d35023-c90e-4a2c-9df6-9c67c9ceefa7" path="/var/lib/kubelet/pods/a8d35023-c90e-4a2c-9df6-9c67c9ceefa7/volumes" Sep 11 12:18:30.928910 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:30.928883 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-af081c-74b8976d5c-tvm6b" Sep 11 12:18:31.058475 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:31.058437 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96a0d2ef-39dd-476e-a527-53e9af9cca98-plugins-config-volume\") pod \"96a0d2ef-39dd-476e-a527-53e9af9cca98\" (UID: \"96a0d2ef-39dd-476e-a527-53e9af9cca98\") " Sep 11 12:18:31.058676 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:31.058511 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqfvc\" (UniqueName: \"kubernetes.io/projected/96a0d2ef-39dd-476e-a527-53e9af9cca98-kube-api-access-wqfvc\") pod \"96a0d2ef-39dd-476e-a527-53e9af9cca98\" (UID: \"96a0d2ef-39dd-476e-a527-53e9af9cca98\") " Sep 11 12:18:31.058830 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:31.058805 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96a0d2ef-39dd-476e-a527-53e9af9cca98-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "96a0d2ef-39dd-476e-a527-53e9af9cca98" (UID: "96a0d2ef-39dd-476e-a527-53e9af9cca98"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 12:18:31.060706 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:31.060679 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96a0d2ef-39dd-476e-a527-53e9af9cca98-kube-api-access-wqfvc" (OuterVolumeSpecName: "kube-api-access-wqfvc") pod "96a0d2ef-39dd-476e-a527-53e9af9cca98" (UID: "96a0d2ef-39dd-476e-a527-53e9af9cca98"). InnerVolumeSpecName "kube-api-access-wqfvc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:18:31.159330 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:31.159284 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96a0d2ef-39dd-476e-a527-53e9af9cca98-plugins-config-volume\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:18:31.159330 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:31.159324 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqfvc\" (UniqueName: \"kubernetes.io/projected/96a0d2ef-39dd-476e-a527-53e9af9cca98-kube-api-access-wqfvc\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:18:31.250137 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:31.250039 2566 generic.go:356] "Generic (PLEG): container finished" podID="96a0d2ef-39dd-476e-a527-53e9af9cca98" containerID="f9e69622b81d1cd118d4e0bf38c64eecf6368e78f0228a9bbf3259522838d61a" exitCode=0 Sep 11 12:18:31.250137 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:31.250088 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-af081c-74b8976d5c-tvm6b" Sep 11 12:18:31.250317 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:31.250130 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-af081c-74b8976d5c-tvm6b" event={"ID":"96a0d2ef-39dd-476e-a527-53e9af9cca98","Type":"ContainerDied","Data":"f9e69622b81d1cd118d4e0bf38c64eecf6368e78f0228a9bbf3259522838d61a"} Sep 11 12:18:31.250317 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:31.250190 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-af081c-74b8976d5c-tvm6b" event={"ID":"96a0d2ef-39dd-476e-a527-53e9af9cca98","Type":"ContainerDied","Data":"9970ddee2ac26168766c0473d272a31e6f12f28ba644bca7f2909996fc740f0c"} Sep 11 12:18:31.250317 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:31.250216 2566 scope.go:117] "RemoveContainer" containerID="f9e69622b81d1cd118d4e0bf38c64eecf6368e78f0228a9bbf3259522838d61a" Sep 11 12:18:31.259499 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:31.259480 2566 scope.go:117] "RemoveContainer" containerID="f9e69622b81d1cd118d4e0bf38c64eecf6368e78f0228a9bbf3259522838d61a" Sep 11 12:18:31.259811 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:18:31.259788 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9e69622b81d1cd118d4e0bf38c64eecf6368e78f0228a9bbf3259522838d61a\": container with ID starting with f9e69622b81d1cd118d4e0bf38c64eecf6368e78f0228a9bbf3259522838d61a not found: ID does not exist" containerID="f9e69622b81d1cd118d4e0bf38c64eecf6368e78f0228a9bbf3259522838d61a" Sep 11 12:18:31.259961 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:31.259820 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9e69622b81d1cd118d4e0bf38c64eecf6368e78f0228a9bbf3259522838d61a"} err="failed to get container status \"f9e69622b81d1cd118d4e0bf38c64eecf6368e78f0228a9bbf3259522838d61a\": rpc error: code = NotFound desc = could not find container \"f9e69622b81d1cd118d4e0bf38c64eecf6368e78f0228a9bbf3259522838d61a\": container with ID starting with f9e69622b81d1cd118d4e0bf38c64eecf6368e78f0228a9bbf3259522838d61a not found: ID does not exist" Sep 11 12:18:31.273901 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:31.273861 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-af081c-74b8976d5c-tvm6b"] Sep 11 12:18:31.277286 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:31.277259 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-af081c-74b8976d5c-tvm6b"] Sep 11 12:18:32.759786 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:32.759754 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96a0d2ef-39dd-476e-a527-53e9af9cca98" path="/var/lib/kubelet/pods/96a0d2ef-39dd-476e-a527-53e9af9cca98/volumes" Sep 11 12:18:37.443400 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:37.443363 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz"] Sep 11 12:18:37.443895 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:37.443639 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" podUID="47192df9-4559-4286-a94e-1f01440c4c65" containerName="istio-proxy" containerID="cri-o://5f4f5fa08fcea6e38307b6572e87f6600899c99aac57f6cc6eb6b870717b05e9" gracePeriod=30 Sep 11 12:18:42.699633 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:42.699608 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" Sep 11 12:18:42.862825 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:42.862792 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47192df9-4559-4286-a94e-1f01440c4c65-workload-socket\") pod \"47192df9-4559-4286-a94e-1f01440c4c65\" (UID: \"47192df9-4559-4286-a94e-1f01440c4c65\") " Sep 11 12:18:42.863038 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:42.862846 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47192df9-4559-4286-a94e-1f01440c4c65-credential-socket\") pod \"47192df9-4559-4286-a94e-1f01440c4c65\" (UID: \"47192df9-4559-4286-a94e-1f01440c4c65\") " Sep 11 12:18:42.863038 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:42.862887 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47192df9-4559-4286-a94e-1f01440c4c65-istio-podinfo\") pod \"47192df9-4559-4286-a94e-1f01440c4c65\" (UID: \"47192df9-4559-4286-a94e-1f01440c4c65\") " Sep 11 12:18:42.863038 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:42.862927 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47192df9-4559-4286-a94e-1f01440c4c65-istio-token\") pod \"47192df9-4559-4286-a94e-1f01440c4c65\" (UID: \"47192df9-4559-4286-a94e-1f01440c4c65\") " Sep 11 12:18:42.863038 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:42.862979 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qcvsd\" (UniqueName: \"kubernetes.io/projected/47192df9-4559-4286-a94e-1f01440c4c65-kube-api-access-qcvsd\") pod \"47192df9-4559-4286-a94e-1f01440c4c65\" (UID: \"47192df9-4559-4286-a94e-1f01440c4c65\") " Sep 11 12:18:42.863038 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:42.863002 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47192df9-4559-4286-a94e-1f01440c4c65-istio-data\") pod \"47192df9-4559-4286-a94e-1f01440c4c65\" (UID: \"47192df9-4559-4286-a94e-1f01440c4c65\") " Sep 11 12:18:42.863038 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:42.863024 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47192df9-4559-4286-a94e-1f01440c4c65-istio-envoy\") pod \"47192df9-4559-4286-a94e-1f01440c4c65\" (UID: \"47192df9-4559-4286-a94e-1f01440c4c65\") " Sep 11 12:18:42.863353 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:42.863065 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47192df9-4559-4286-a94e-1f01440c4c65-workload-certs\") pod \"47192df9-4559-4286-a94e-1f01440c4c65\" (UID: \"47192df9-4559-4286-a94e-1f01440c4c65\") " Sep 11 12:18:42.863353 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:42.863091 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47192df9-4559-4286-a94e-1f01440c4c65-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "47192df9-4559-4286-a94e-1f01440c4c65" (UID: "47192df9-4559-4286-a94e-1f01440c4c65"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:18:42.863353 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:42.863150 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47192df9-4559-4286-a94e-1f01440c4c65-istiod-ca-cert\") pod \"47192df9-4559-4286-a94e-1f01440c4c65\" (UID: \"47192df9-4559-4286-a94e-1f01440c4c65\") " Sep 11 12:18:42.863353 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:42.863330 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47192df9-4559-4286-a94e-1f01440c4c65-istio-data" (OuterVolumeSpecName: "istio-data") pod "47192df9-4559-4286-a94e-1f01440c4c65" (UID: "47192df9-4559-4286-a94e-1f01440c4c65"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:18:42.863353 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:42.863346 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47192df9-4559-4286-a94e-1f01440c4c65-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "47192df9-4559-4286-a94e-1f01440c4c65" (UID: "47192df9-4559-4286-a94e-1f01440c4c65"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:18:42.863600 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:42.863492 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47192df9-4559-4286-a94e-1f01440c4c65-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "47192df9-4559-4286-a94e-1f01440c4c65" (UID: "47192df9-4559-4286-a94e-1f01440c4c65"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:18:42.863600 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:42.863542 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47192df9-4559-4286-a94e-1f01440c4c65-workload-certs\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:18:42.863600 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:42.863562 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47192df9-4559-4286-a94e-1f01440c4c65-workload-socket\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:18:42.863600 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:42.863577 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47192df9-4559-4286-a94e-1f01440c4c65-istio-data\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:18:42.863811 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:42.863642 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47192df9-4559-4286-a94e-1f01440c4c65-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "47192df9-4559-4286-a94e-1f01440c4c65" (UID: "47192df9-4559-4286-a94e-1f01440c4c65"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 12:18:42.865309 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:42.865270 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47192df9-4559-4286-a94e-1f01440c4c65-istio-token" (OuterVolumeSpecName: "istio-token") pod "47192df9-4559-4286-a94e-1f01440c4c65" (UID: "47192df9-4559-4286-a94e-1f01440c4c65"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:18:42.865427 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:42.865308 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47192df9-4559-4286-a94e-1f01440c4c65-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "47192df9-4559-4286-a94e-1f01440c4c65" (UID: "47192df9-4559-4286-a94e-1f01440c4c65"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:18:42.865539 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:42.865524 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/47192df9-4559-4286-a94e-1f01440c4c65-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "47192df9-4559-4286-a94e-1f01440c4c65" (UID: "47192df9-4559-4286-a94e-1f01440c4c65"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 11 12:18:42.865582 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:42.865540 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47192df9-4559-4286-a94e-1f01440c4c65-kube-api-access-qcvsd" (OuterVolumeSpecName: "kube-api-access-qcvsd") pod "47192df9-4559-4286-a94e-1f01440c4c65" (UID: "47192df9-4559-4286-a94e-1f01440c4c65"). InnerVolumeSpecName "kube-api-access-qcvsd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:18:42.964145 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:42.964107 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qcvsd\" (UniqueName: \"kubernetes.io/projected/47192df9-4559-4286-a94e-1f01440c4c65-kube-api-access-qcvsd\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:18:42.964145 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:42.964139 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47192df9-4559-4286-a94e-1f01440c4c65-istio-envoy\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:18:42.964145 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:42.964149 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47192df9-4559-4286-a94e-1f01440c4c65-istiod-ca-cert\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:18:42.964145 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:42.964157 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47192df9-4559-4286-a94e-1f01440c4c65-credential-socket\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:18:42.964419 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:42.964166 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47192df9-4559-4286-a94e-1f01440c4c65-istio-podinfo\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:18:42.964419 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:42.964174 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47192df9-4559-4286-a94e-1f01440c4c65-istio-token\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:18:43.308702 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:43.308617 2566 generic.go:356] "Generic (PLEG): container finished" podID="47192df9-4559-4286-a94e-1f01440c4c65" containerID="5f4f5fa08fcea6e38307b6572e87f6600899c99aac57f6cc6eb6b870717b05e9" exitCode=0 Sep 11 12:18:43.308702 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:43.308666 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" Sep 11 12:18:43.308913 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:43.308701 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" event={"ID":"47192df9-4559-4286-a94e-1f01440c4c65","Type":"ContainerDied","Data":"5f4f5fa08fcea6e38307b6572e87f6600899c99aac57f6cc6eb6b870717b05e9"} Sep 11 12:18:43.308913 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:43.308754 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz" event={"ID":"47192df9-4559-4286-a94e-1f01440c4c65","Type":"ContainerDied","Data":"98b6c7d39e30ff1e372b2fc9c4689b6b498bb8b8e6354ae7c2c7a78a9bf787dc"} Sep 11 12:18:43.308913 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:43.308777 2566 scope.go:117] "RemoveContainer" containerID="5f4f5fa08fcea6e38307b6572e87f6600899c99aac57f6cc6eb6b870717b05e9" Sep 11 12:18:43.317643 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:43.317627 2566 scope.go:117] "RemoveContainer" containerID="f2ad890a0a45f606d2171a404cd28259f497956d3dab85c93803c892682aaa90" Sep 11 12:18:43.325322 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:43.325305 2566 scope.go:117] "RemoveContainer" containerID="5f4f5fa08fcea6e38307b6572e87f6600899c99aac57f6cc6eb6b870717b05e9" Sep 11 12:18:43.325595 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:18:43.325561 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f4f5fa08fcea6e38307b6572e87f6600899c99aac57f6cc6eb6b870717b05e9\": container with ID starting with 5f4f5fa08fcea6e38307b6572e87f6600899c99aac57f6cc6eb6b870717b05e9 not found: ID does not exist" containerID="5f4f5fa08fcea6e38307b6572e87f6600899c99aac57f6cc6eb6b870717b05e9" Sep 11 12:18:43.325645 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:43.325604 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f4f5fa08fcea6e38307b6572e87f6600899c99aac57f6cc6eb6b870717b05e9"} err="failed to get container status \"5f4f5fa08fcea6e38307b6572e87f6600899c99aac57f6cc6eb6b870717b05e9\": rpc error: code = NotFound desc = could not find container \"5f4f5fa08fcea6e38307b6572e87f6600899c99aac57f6cc6eb6b870717b05e9\": container with ID starting with 5f4f5fa08fcea6e38307b6572e87f6600899c99aac57f6cc6eb6b870717b05e9 not found: ID does not exist" Sep 11 12:18:43.325645 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:43.325622 2566 scope.go:117] "RemoveContainer" containerID="f2ad890a0a45f606d2171a404cd28259f497956d3dab85c93803c892682aaa90" Sep 11 12:18:43.325851 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:18:43.325834 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2ad890a0a45f606d2171a404cd28259f497956d3dab85c93803c892682aaa90\": container with ID starting with f2ad890a0a45f606d2171a404cd28259f497956d3dab85c93803c892682aaa90 not found: ID does not exist" containerID="f2ad890a0a45f606d2171a404cd28259f497956d3dab85c93803c892682aaa90" Sep 11 12:18:43.325928 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:43.325860 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2ad890a0a45f606d2171a404cd28259f497956d3dab85c93803c892682aaa90"} err="failed to get container status \"f2ad890a0a45f606d2171a404cd28259f497956d3dab85c93803c892682aaa90\": rpc error: code = NotFound desc = could not find container \"f2ad890a0a45f606d2171a404cd28259f497956d3dab85c93803c892682aaa90\": container with ID starting with f2ad890a0a45f606d2171a404cd28259f497956d3dab85c93803c892682aaa90 not found: ID does not exist" Sep 11 12:18:43.329708 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:43.329686 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz"] Sep 11 12:18:43.335890 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:43.335856 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-af081c-openshift-default-547c47967d-b6qcz"] Sep 11 12:18:44.759209 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:44.759178 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47192df9-4559-4286-a94e-1f01440c4c65" path="/var/lib/kubelet/pods/47192df9-4559-4286-a94e-1f01440c4c65/volumes" Sep 11 12:18:48.496217 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:48.496177 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d844cc-cc8b44746-lzlgq"] Sep 11 12:18:48.496705 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:48.496399 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d844cc-cc8b44746-lzlgq" podUID="cc86f6e6-bbf2-43d6-ae13-16b3a1484b92" containerName="payload-pre-processing" containerID="cri-o://13df99bc8a44cc3aa94fcc19f381566c71956074aad684604db99012887ff9a9" gracePeriod=30 Sep 11 12:18:48.742585 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:48.742558 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d844cc-cc8b44746-lzlgq" Sep 11 12:18:48.911312 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:48.911280 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc86f6e6-bbf2-43d6-ae13-16b3a1484b92-plugins-config-volume\") pod \"cc86f6e6-bbf2-43d6-ae13-16b3a1484b92\" (UID: \"cc86f6e6-bbf2-43d6-ae13-16b3a1484b92\") " Sep 11 12:18:48.911312 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:48.911314 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-whrpf\" (UniqueName: \"kubernetes.io/projected/cc86f6e6-bbf2-43d6-ae13-16b3a1484b92-kube-api-access-whrpf\") pod \"cc86f6e6-bbf2-43d6-ae13-16b3a1484b92\" (UID: \"cc86f6e6-bbf2-43d6-ae13-16b3a1484b92\") " Sep 11 12:18:48.911680 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:48.911654 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc86f6e6-bbf2-43d6-ae13-16b3a1484b92-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cc86f6e6-bbf2-43d6-ae13-16b3a1484b92" (UID: "cc86f6e6-bbf2-43d6-ae13-16b3a1484b92"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 12:18:48.913424 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:48.913393 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc86f6e6-bbf2-43d6-ae13-16b3a1484b92-kube-api-access-whrpf" (OuterVolumeSpecName: "kube-api-access-whrpf") pod "cc86f6e6-bbf2-43d6-ae13-16b3a1484b92" (UID: "cc86f6e6-bbf2-43d6-ae13-16b3a1484b92"). InnerVolumeSpecName "kube-api-access-whrpf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:18:48.984370 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:48.984336 2566 scope.go:117] "RemoveContainer" containerID="13df99bc8a44cc3aa94fcc19f381566c71956074aad684604db99012887ff9a9" Sep 11 12:18:49.012176 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:49.012148 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc86f6e6-bbf2-43d6-ae13-16b3a1484b92-plugins-config-volume\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:18:49.012176 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:49.012174 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whrpf\" (UniqueName: \"kubernetes.io/projected/cc86f6e6-bbf2-43d6-ae13-16b3a1484b92-kube-api-access-whrpf\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:18:49.336493 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:49.336463 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d844cc-cc8b44746-lzlgq" Sep 11 12:18:49.336695 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:49.336463 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d844cc-cc8b44746-lzlgq" event={"ID":"cc86f6e6-bbf2-43d6-ae13-16b3a1484b92","Type":"ContainerDied","Data":"13df99bc8a44cc3aa94fcc19f381566c71956074aad684604db99012887ff9a9"} Sep 11 12:18:49.336695 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:49.336624 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d844cc-cc8b44746-lzlgq" event={"ID":"cc86f6e6-bbf2-43d6-ae13-16b3a1484b92","Type":"ContainerDied","Data":"e9c80081afeb3bb572f1b5bfcf6936d0e5beda40d20d7beebb7fa89ece39a41a"} Sep 11 12:18:49.372145 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:49.372109 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d844cc-cc8b44746-lzlgq"] Sep 11 12:18:49.374060 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:49.374036 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d844cc-cc8b44746-lzlgq"] Sep 11 12:18:50.757605 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:18:50.757567 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc86f6e6-bbf2-43d6-ae13-16b3a1484b92" path="/var/lib/kubelet/pods/cc86f6e6-bbf2-43d6-ae13-16b3a1484b92/volumes" Sep 11 12:20:00.114425 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.114387 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-c4fd66f5c-zvt24"] Sep 11 12:20:00.115070 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.114816 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47192df9-4559-4286-a94e-1f01440c4c65" containerName="istio-proxy" Sep 11 12:20:00.115070 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.114835 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="47192df9-4559-4286-a94e-1f01440c4c65" containerName="istio-proxy" Sep 11 12:20:00.115070 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.114848 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6dd23c7-c2ad-4620-a40e-de06161ee314" containerName="istio-proxy" Sep 11 12:20:00.115070 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.114856 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6dd23c7-c2ad-4620-a40e-de06161ee314" containerName="istio-proxy" Sep 11 12:20:00.115070 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.114885 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47192df9-4559-4286-a94e-1f01440c4c65" containerName="istio-proxy" Sep 11 12:20:00.115070 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.114897 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="47192df9-4559-4286-a94e-1f01440c4c65" containerName="istio-proxy" Sep 11 12:20:00.115070 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.114907 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44fb82a8-a515-4ff1-9684-b2eefcf50cb1" containerName="revoke-keys" Sep 11 12:20:00.115070 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.114915 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="44fb82a8-a515-4ff1-9684-b2eefcf50cb1" containerName="revoke-keys" Sep 11 12:20:00.115070 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.114939 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96a0d2ef-39dd-476e-a527-53e9af9cca98" containerName="payload-processing" Sep 11 12:20:00.115070 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.114948 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="96a0d2ef-39dd-476e-a527-53e9af9cca98" containerName="payload-processing" Sep 11 12:20:00.115070 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.114961 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0" containerName="istio-proxy" Sep 11 12:20:00.115070 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.114969 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0" containerName="istio-proxy" Sep 11 12:20:00.115070 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.114982 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74844184-6bfc-476f-b940-14bbb7018f24" containerName="payload-processing" Sep 11 12:20:00.115070 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.114990 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="74844184-6bfc-476f-b940-14bbb7018f24" containerName="payload-processing" Sep 11 12:20:00.115070 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.115002 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecc4a81b-a2ac-4661-9dfc-eda838871e02" containerName="payload-pre-processing" Sep 11 12:20:00.115070 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.115011 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecc4a81b-a2ac-4661-9dfc-eda838871e02" containerName="payload-pre-processing" Sep 11 12:20:00.115070 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.115023 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0" containerName="istio-proxy" Sep 11 12:20:00.115070 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.115031 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0" containerName="istio-proxy" Sep 11 12:20:00.115070 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.115070 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc86f6e6-bbf2-43d6-ae13-16b3a1484b92" containerName="payload-pre-processing" Sep 11 12:20:00.115070 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.115079 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc86f6e6-bbf2-43d6-ae13-16b3a1484b92" containerName="payload-pre-processing" Sep 11 12:20:00.116059 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.115091 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a74ee09-e4aa-4e74-a3db-b7aab95dca04" containerName="payload-processing" Sep 11 12:20:00.116059 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.115099 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a74ee09-e4aa-4e74-a3db-b7aab95dca04" containerName="payload-processing" Sep 11 12:20:00.116059 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.115202 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="47192df9-4559-4286-a94e-1f01440c4c65" containerName="istio-proxy" Sep 11 12:20:00.116059 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.115215 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="47192df9-4559-4286-a94e-1f01440c4c65" containerName="istio-proxy" Sep 11 12:20:00.116059 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.115224 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="74844184-6bfc-476f-b940-14bbb7018f24" containerName="payload-processing" Sep 11 12:20:00.116059 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.115235 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0" containerName="istio-proxy" Sep 11 12:20:00.116059 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.115243 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="44fb82a8-a515-4ff1-9684-b2eefcf50cb1" containerName="revoke-keys" Sep 11 12:20:00.116059 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.115253 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="96a0d2ef-39dd-476e-a527-53e9af9cca98" containerName="payload-processing" Sep 11 12:20:00.116059 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.115260 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a74ee09-e4aa-4e74-a3db-b7aab95dca04" containerName="payload-processing" Sep 11 12:20:00.116059 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.115268 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecc4a81b-a2ac-4661-9dfc-eda838871e02" containerName="payload-pre-processing" Sep 11 12:20:00.116059 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.115277 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="33a4cd0e-67ae-4baa-8551-d1a6c21b2cd0" containerName="istio-proxy" Sep 11 12:20:00.116059 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.115284 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6dd23c7-c2ad-4620-a40e-de06161ee314" containerName="istio-proxy" Sep 11 12:20:00.116059 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.115294 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc86f6e6-bbf2-43d6-ae13-16b3a1484b92" containerName="payload-pre-processing" Sep 11 12:20:00.116059 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.115303 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6dd23c7-c2ad-4620-a40e-de06161ee314" containerName="istio-proxy" Sep 11 12:20:00.118293 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.118272 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c4fd66f5c-zvt24" Sep 11 12:20:00.121885 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.121847 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 11 12:20:00.121885 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.121861 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 11 12:20:00.122058 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.121855 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-qmtgj\"" Sep 11 12:20:00.122058 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.121853 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 11 12:20:00.131160 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.131133 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-c4fd66f5c-zvt24"] Sep 11 12:20:00.209538 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.209502 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f1c692d-dad2-4733-99ad-ddd6c6d6df1f-metrics-certs\") pod \"maas-controller-c4fd66f5c-zvt24\" (UID: \"0f1c692d-dad2-4733-99ad-ddd6c6d6df1f\") " pod="opendatahub/maas-controller-c4fd66f5c-zvt24" Sep 11 12:20:00.209716 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.209553 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxqsp\" (UniqueName: \"kubernetes.io/projected/0f1c692d-dad2-4733-99ad-ddd6c6d6df1f-kube-api-access-rxqsp\") pod \"maas-controller-c4fd66f5c-zvt24\" (UID: \"0f1c692d-dad2-4733-99ad-ddd6c6d6df1f\") " pod="opendatahub/maas-controller-c4fd66f5c-zvt24" Sep 11 12:20:00.209716 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.209646 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0f1c692d-dad2-4733-99ad-ddd6c6d6df1f-webhook-cert\") pod \"maas-controller-c4fd66f5c-zvt24\" (UID: \"0f1c692d-dad2-4733-99ad-ddd6c6d6df1f\") " pod="opendatahub/maas-controller-c4fd66f5c-zvt24" Sep 11 12:20:00.310365 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.310330 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f1c692d-dad2-4733-99ad-ddd6c6d6df1f-metrics-certs\") pod \"maas-controller-c4fd66f5c-zvt24\" (UID: \"0f1c692d-dad2-4733-99ad-ddd6c6d6df1f\") " pod="opendatahub/maas-controller-c4fd66f5c-zvt24" Sep 11 12:20:00.310517 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.310373 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxqsp\" (UniqueName: \"kubernetes.io/projected/0f1c692d-dad2-4733-99ad-ddd6c6d6df1f-kube-api-access-rxqsp\") pod \"maas-controller-c4fd66f5c-zvt24\" (UID: \"0f1c692d-dad2-4733-99ad-ddd6c6d6df1f\") " pod="opendatahub/maas-controller-c4fd66f5c-zvt24" Sep 11 12:20:00.310517 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.310407 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0f1c692d-dad2-4733-99ad-ddd6c6d6df1f-webhook-cert\") pod \"maas-controller-c4fd66f5c-zvt24\" (UID: \"0f1c692d-dad2-4733-99ad-ddd6c6d6df1f\") " pod="opendatahub/maas-controller-c4fd66f5c-zvt24" Sep 11 12:20:00.312899 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.312856 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f1c692d-dad2-4733-99ad-ddd6c6d6df1f-metrics-certs\") pod \"maas-controller-c4fd66f5c-zvt24\" (UID: \"0f1c692d-dad2-4733-99ad-ddd6c6d6df1f\") " pod="opendatahub/maas-controller-c4fd66f5c-zvt24" Sep 11 12:20:00.312899 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.312888 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0f1c692d-dad2-4733-99ad-ddd6c6d6df1f-webhook-cert\") pod \"maas-controller-c4fd66f5c-zvt24\" (UID: \"0f1c692d-dad2-4733-99ad-ddd6c6d6df1f\") " pod="opendatahub/maas-controller-c4fd66f5c-zvt24" Sep 11 12:20:00.321286 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.321266 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxqsp\" (UniqueName: \"kubernetes.io/projected/0f1c692d-dad2-4733-99ad-ddd6c6d6df1f-kube-api-access-rxqsp\") pod \"maas-controller-c4fd66f5c-zvt24\" (UID: \"0f1c692d-dad2-4733-99ad-ddd6c6d6df1f\") " pod="opendatahub/maas-controller-c4fd66f5c-zvt24" Sep 11 12:20:00.428578 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.428505 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c4fd66f5c-zvt24" Sep 11 12:20:00.701694 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.701621 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-c4fd66f5c-zvt24"] Sep 11 12:20:00.704672 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:20:00.704646 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f1c692d_dad2_4733_99ad_ddd6c6d6df1f.slice/crio-d5fd78e3ef64a6f75bfd3916c35667a169958b5f14f7414b186498581b622634 WatchSource:0}: Error finding container d5fd78e3ef64a6f75bfd3916c35667a169958b5f14f7414b186498581b622634: Status 404 returned error can't find the container with id d5fd78e3ef64a6f75bfd3916c35667a169958b5f14f7414b186498581b622634 Sep 11 12:20:00.705929 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:00.705911 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 11 12:20:01.670974 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:01.670937 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c4fd66f5c-zvt24" event={"ID":"0f1c692d-dad2-4733-99ad-ddd6c6d6df1f","Type":"ContainerStarted","Data":"bc1cdc2b6ab37cd5964b2fa69999fee00ede9783c22714243d7ea5a4a969421b"} Sep 11 12:20:01.671348 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:01.670994 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c4fd66f5c-zvt24" event={"ID":"0f1c692d-dad2-4733-99ad-ddd6c6d6df1f","Type":"ContainerStarted","Data":"d5fd78e3ef64a6f75bfd3916c35667a169958b5f14f7414b186498581b622634"} Sep 11 12:20:01.671348 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:01.671015 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-c4fd66f5c-zvt24" Sep 11 12:20:01.691335 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:01.691287 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-c4fd66f5c-zvt24" podStartSLOduration=0.885491827 podStartE2EDuration="1.691272295s" totalImagesPullingTime="805.780468ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:20:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:20:01.690694084 +0000 UTC m=+1933.455667258" watchObservedRunningTime="2026-09-11 12:20:01.691272295 +0000 UTC m=+1933.456245486" Sep 11 12:20:12.677952 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:12.677916 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-c4fd66f5c-zvt24" Sep 11 12:20:22.077547 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:22.077513 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-c4fd66f5c-zvt24"] Sep 11 12:20:22.077953 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:22.077733 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-c4fd66f5c-zvt24" podUID="0f1c692d-dad2-4733-99ad-ddd6c6d6df1f" containerName="manager" containerID="cri-o://bc1cdc2b6ab37cd5964b2fa69999fee00ede9783c22714243d7ea5a4a969421b" gracePeriod=30 Sep 11 12:20:22.324442 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:22.324415 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c4fd66f5c-zvt24" Sep 11 12:20:22.403064 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:22.402974 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rxqsp\" (UniqueName: \"kubernetes.io/projected/0f1c692d-dad2-4733-99ad-ddd6c6d6df1f-kube-api-access-rxqsp\") pod \"0f1c692d-dad2-4733-99ad-ddd6c6d6df1f\" (UID: \"0f1c692d-dad2-4733-99ad-ddd6c6d6df1f\") " Sep 11 12:20:22.403064 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:22.403038 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f1c692d-dad2-4733-99ad-ddd6c6d6df1f-metrics-certs\") pod \"0f1c692d-dad2-4733-99ad-ddd6c6d6df1f\" (UID: \"0f1c692d-dad2-4733-99ad-ddd6c6d6df1f\") " Sep 11 12:20:22.403299 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:22.403223 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0f1c692d-dad2-4733-99ad-ddd6c6d6df1f-webhook-cert\") pod \"0f1c692d-dad2-4733-99ad-ddd6c6d6df1f\" (UID: \"0f1c692d-dad2-4733-99ad-ddd6c6d6df1f\") " Sep 11 12:20:22.405177 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:22.405142 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f1c692d-dad2-4733-99ad-ddd6c6d6df1f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0f1c692d-dad2-4733-99ad-ddd6c6d6df1f" (UID: "0f1c692d-dad2-4733-99ad-ddd6c6d6df1f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 12:20:22.405278 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:22.405177 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f1c692d-dad2-4733-99ad-ddd6c6d6df1f-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "0f1c692d-dad2-4733-99ad-ddd6c6d6df1f" (UID: "0f1c692d-dad2-4733-99ad-ddd6c6d6df1f"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 12:20:22.405470 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:22.405451 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f1c692d-dad2-4733-99ad-ddd6c6d6df1f-kube-api-access-rxqsp" (OuterVolumeSpecName: "kube-api-access-rxqsp") pod "0f1c692d-dad2-4733-99ad-ddd6c6d6df1f" (UID: "0f1c692d-dad2-4733-99ad-ddd6c6d6df1f"). InnerVolumeSpecName "kube-api-access-rxqsp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:20:22.504516 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:22.504483 2566 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0f1c692d-dad2-4733-99ad-ddd6c6d6df1f-webhook-cert\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:20:22.504683 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:22.504511 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxqsp\" (UniqueName: \"kubernetes.io/projected/0f1c692d-dad2-4733-99ad-ddd6c6d6df1f-kube-api-access-rxqsp\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:20:22.504683 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:22.504536 2566 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f1c692d-dad2-4733-99ad-ddd6c6d6df1f-metrics-certs\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:20:22.766152 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:22.766120 2566 generic.go:356] "Generic (PLEG): container finished" podID="0f1c692d-dad2-4733-99ad-ddd6c6d6df1f" containerID="bc1cdc2b6ab37cd5964b2fa69999fee00ede9783c22714243d7ea5a4a969421b" exitCode=0 Sep 11 12:20:22.766286 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:22.766150 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c4fd66f5c-zvt24" event={"ID":"0f1c692d-dad2-4733-99ad-ddd6c6d6df1f","Type":"ContainerDied","Data":"bc1cdc2b6ab37cd5964b2fa69999fee00ede9783c22714243d7ea5a4a969421b"} Sep 11 12:20:22.766286 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:22.766167 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c4fd66f5c-zvt24" Sep 11 12:20:22.766286 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:22.766201 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c4fd66f5c-zvt24" event={"ID":"0f1c692d-dad2-4733-99ad-ddd6c6d6df1f","Type":"ContainerDied","Data":"d5fd78e3ef64a6f75bfd3916c35667a169958b5f14f7414b186498581b622634"} Sep 11 12:20:22.766286 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:22.766224 2566 scope.go:117] "RemoveContainer" containerID="bc1cdc2b6ab37cd5964b2fa69999fee00ede9783c22714243d7ea5a4a969421b" Sep 11 12:20:22.774546 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:22.774517 2566 scope.go:117] "RemoveContainer" containerID="bc1cdc2b6ab37cd5964b2fa69999fee00ede9783c22714243d7ea5a4a969421b" Sep 11 12:20:22.774800 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:20:22.774781 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc1cdc2b6ab37cd5964b2fa69999fee00ede9783c22714243d7ea5a4a969421b\": container with ID starting with bc1cdc2b6ab37cd5964b2fa69999fee00ede9783c22714243d7ea5a4a969421b not found: ID does not exist" containerID="bc1cdc2b6ab37cd5964b2fa69999fee00ede9783c22714243d7ea5a4a969421b" Sep 11 12:20:22.774880 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:22.774812 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc1cdc2b6ab37cd5964b2fa69999fee00ede9783c22714243d7ea5a4a969421b"} err="failed to get container status \"bc1cdc2b6ab37cd5964b2fa69999fee00ede9783c22714243d7ea5a4a969421b\": rpc error: code = NotFound desc = could not find container \"bc1cdc2b6ab37cd5964b2fa69999fee00ede9783c22714243d7ea5a4a969421b\": container with ID starting with bc1cdc2b6ab37cd5964b2fa69999fee00ede9783c22714243d7ea5a4a969421b not found: ID does not exist" Sep 11 12:20:22.784905 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:22.784880 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-c4fd66f5c-zvt24"] Sep 11 12:20:22.789757 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:22.789733 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-c4fd66f5c-zvt24"] Sep 11 12:20:23.783094 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:23.783061 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-c4fd66f5c-lnlwl"] Sep 11 12:20:23.783461 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:23.783374 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6dd23c7-c2ad-4620-a40e-de06161ee314" containerName="istio-proxy" Sep 11 12:20:23.783461 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:23.783390 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6dd23c7-c2ad-4620-a40e-de06161ee314" containerName="istio-proxy" Sep 11 12:20:23.783461 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:23.783416 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f1c692d-dad2-4733-99ad-ddd6c6d6df1f" containerName="manager" Sep 11 12:20:23.783461 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:23.783421 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f1c692d-dad2-4733-99ad-ddd6c6d6df1f" containerName="manager" Sep 11 12:20:23.783605 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:23.783493 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f1c692d-dad2-4733-99ad-ddd6c6d6df1f" containerName="manager" Sep 11 12:20:23.787514 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:23.787497 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c4fd66f5c-lnlwl" Sep 11 12:20:23.796115 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:23.796092 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 11 12:20:23.796319 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:23.796303 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 11 12:20:23.796974 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:23.796960 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-qmtgj\"" Sep 11 12:20:23.797030 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:23.797002 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 11 12:20:23.810398 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:23.810370 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-c4fd66f5c-lnlwl"] Sep 11 12:20:23.918003 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:23.917962 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/df5f488c-1c66-462d-a1c3-b8abe43dd74f-webhook-cert\") pod \"maas-controller-c4fd66f5c-lnlwl\" (UID: \"df5f488c-1c66-462d-a1c3-b8abe43dd74f\") " pod="opendatahub/maas-controller-c4fd66f5c-lnlwl" Sep 11 12:20:23.918144 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:23.918012 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqkv9\" (UniqueName: \"kubernetes.io/projected/df5f488c-1c66-462d-a1c3-b8abe43dd74f-kube-api-access-sqkv9\") pod \"maas-controller-c4fd66f5c-lnlwl\" (UID: \"df5f488c-1c66-462d-a1c3-b8abe43dd74f\") " pod="opendatahub/maas-controller-c4fd66f5c-lnlwl" Sep 11 12:20:23.918144 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:23.918081 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df5f488c-1c66-462d-a1c3-b8abe43dd74f-metrics-certs\") pod \"maas-controller-c4fd66f5c-lnlwl\" (UID: \"df5f488c-1c66-462d-a1c3-b8abe43dd74f\") " pod="opendatahub/maas-controller-c4fd66f5c-lnlwl" Sep 11 12:20:24.019320 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:24.019281 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/df5f488c-1c66-462d-a1c3-b8abe43dd74f-webhook-cert\") pod \"maas-controller-c4fd66f5c-lnlwl\" (UID: \"df5f488c-1c66-462d-a1c3-b8abe43dd74f\") " pod="opendatahub/maas-controller-c4fd66f5c-lnlwl" Sep 11 12:20:24.019501 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:24.019329 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqkv9\" (UniqueName: \"kubernetes.io/projected/df5f488c-1c66-462d-a1c3-b8abe43dd74f-kube-api-access-sqkv9\") pod \"maas-controller-c4fd66f5c-lnlwl\" (UID: \"df5f488c-1c66-462d-a1c3-b8abe43dd74f\") " pod="opendatahub/maas-controller-c4fd66f5c-lnlwl" Sep 11 12:20:24.019501 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:24.019362 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df5f488c-1c66-462d-a1c3-b8abe43dd74f-metrics-certs\") pod \"maas-controller-c4fd66f5c-lnlwl\" (UID: \"df5f488c-1c66-462d-a1c3-b8abe43dd74f\") " pod="opendatahub/maas-controller-c4fd66f5c-lnlwl" Sep 11 12:20:24.021803 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:24.021781 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df5f488c-1c66-462d-a1c3-b8abe43dd74f-metrics-certs\") pod \"maas-controller-c4fd66f5c-lnlwl\" (UID: \"df5f488c-1c66-462d-a1c3-b8abe43dd74f\") " pod="opendatahub/maas-controller-c4fd66f5c-lnlwl" Sep 11 12:20:24.021892 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:24.021853 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/df5f488c-1c66-462d-a1c3-b8abe43dd74f-webhook-cert\") pod \"maas-controller-c4fd66f5c-lnlwl\" (UID: \"df5f488c-1c66-462d-a1c3-b8abe43dd74f\") " pod="opendatahub/maas-controller-c4fd66f5c-lnlwl" Sep 11 12:20:24.029442 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:24.029420 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqkv9\" (UniqueName: \"kubernetes.io/projected/df5f488c-1c66-462d-a1c3-b8abe43dd74f-kube-api-access-sqkv9\") pod \"maas-controller-c4fd66f5c-lnlwl\" (UID: \"df5f488c-1c66-462d-a1c3-b8abe43dd74f\") " pod="opendatahub/maas-controller-c4fd66f5c-lnlwl" Sep 11 12:20:24.097309 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:24.097280 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c4fd66f5c-lnlwl" Sep 11 12:20:24.372204 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:24.372112 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-c4fd66f5c-lnlwl"] Sep 11 12:20:24.375019 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:20:24.374992 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf5f488c_1c66_462d_a1c3_b8abe43dd74f.slice/crio-56ad15960898445f707487692380a21cd04030c4a396db8413eef05c7d97169a WatchSource:0}: Error finding container 56ad15960898445f707487692380a21cd04030c4a396db8413eef05c7d97169a: Status 404 returned error can't find the container with id 56ad15960898445f707487692380a21cd04030c4a396db8413eef05c7d97169a Sep 11 12:20:24.758223 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:24.758195 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f1c692d-dad2-4733-99ad-ddd6c6d6df1f" path="/var/lib/kubelet/pods/0f1c692d-dad2-4733-99ad-ddd6c6d6df1f/volumes" Sep 11 12:20:24.778504 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:24.778472 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c4fd66f5c-lnlwl" event={"ID":"df5f488c-1c66-462d-a1c3-b8abe43dd74f","Type":"ContainerStarted","Data":"22770945208b1a4a8dd18a2408eaf796cf30557610fe72771fc5e8cb684a186f"} Sep 11 12:20:24.778653 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:24.778533 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c4fd66f5c-lnlwl" event={"ID":"df5f488c-1c66-462d-a1c3-b8abe43dd74f","Type":"ContainerStarted","Data":"56ad15960898445f707487692380a21cd04030c4a396db8413eef05c7d97169a"} Sep 11 12:20:24.778653 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:24.778563 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-c4fd66f5c-lnlwl" Sep 11 12:20:24.801816 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:24.801759 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-c4fd66f5c-lnlwl" podStartSLOduration=1.484256396 podStartE2EDuration="1.80174204s" totalImagesPullingTime="317.485644ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:20:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:20:24.801376903 +0000 UTC m=+1956.566350103" watchObservedRunningTime="2026-09-11 12:20:24.80174204 +0000 UTC m=+1956.566715216" Sep 11 12:20:35.785549 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:20:35.785516 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-c4fd66f5c-lnlwl" Sep 11 12:21:32.140585 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:21:32.140549 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f292q"] Sep 11 12:21:32.141130 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:21:32.140778 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f292q" podUID="16e95dd7-18ae-4528-ab96-79201cf6079c" containerName="manager" containerID="cri-o://1d39d9d15da2b1855fb6dfae71ae9d6c605e912e35b8b86bcbef9833a084eb1c" gracePeriod=10 Sep 11 12:21:32.386775 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:21:32.386748 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f292q" Sep 11 12:21:32.405427 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:21:32.405367 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pnz9w\" (UniqueName: \"kubernetes.io/projected/16e95dd7-18ae-4528-ab96-79201cf6079c-kube-api-access-pnz9w\") pod \"16e95dd7-18ae-4528-ab96-79201cf6079c\" (UID: \"16e95dd7-18ae-4528-ab96-79201cf6079c\") " Sep 11 12:21:32.405427 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:21:32.405417 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/16e95dd7-18ae-4528-ab96-79201cf6079c-extensions-socket-volume\") pod \"16e95dd7-18ae-4528-ab96-79201cf6079c\" (UID: \"16e95dd7-18ae-4528-ab96-79201cf6079c\") " Sep 11 12:21:32.405815 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:21:32.405793 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16e95dd7-18ae-4528-ab96-79201cf6079c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "16e95dd7-18ae-4528-ab96-79201cf6079c" (UID: "16e95dd7-18ae-4528-ab96-79201cf6079c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:21:32.407460 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:21:32.407436 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16e95dd7-18ae-4528-ab96-79201cf6079c-kube-api-access-pnz9w" (OuterVolumeSpecName: "kube-api-access-pnz9w") pod "16e95dd7-18ae-4528-ab96-79201cf6079c" (UID: "16e95dd7-18ae-4528-ab96-79201cf6079c"). InnerVolumeSpecName "kube-api-access-pnz9w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:21:32.506799 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:21:32.506766 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pnz9w\" (UniqueName: \"kubernetes.io/projected/16e95dd7-18ae-4528-ab96-79201cf6079c-kube-api-access-pnz9w\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:21:32.506799 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:21:32.506794 2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/16e95dd7-18ae-4528-ab96-79201cf6079c-extensions-socket-volume\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:21:33.083824 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:21:33.083787 2566 generic.go:356] "Generic (PLEG): container finished" podID="16e95dd7-18ae-4528-ab96-79201cf6079c" containerID="1d39d9d15da2b1855fb6dfae71ae9d6c605e912e35b8b86bcbef9833a084eb1c" exitCode=0 Sep 11 12:21:33.084045 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:21:33.083833 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f292q" Sep 11 12:21:33.084045 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:21:33.083878 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f292q" event={"ID":"16e95dd7-18ae-4528-ab96-79201cf6079c","Type":"ContainerDied","Data":"1d39d9d15da2b1855fb6dfae71ae9d6c605e912e35b8b86bcbef9833a084eb1c"} Sep 11 12:21:33.084045 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:21:33.083972 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f292q" event={"ID":"16e95dd7-18ae-4528-ab96-79201cf6079c","Type":"ContainerDied","Data":"c216b22b26eb45b57f93ca8e884a83509792331eaa3a94104f573e8255e22c8e"} Sep 11 12:21:33.084045 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:21:33.083995 2566 scope.go:117] "RemoveContainer" containerID="1d39d9d15da2b1855fb6dfae71ae9d6c605e912e35b8b86bcbef9833a084eb1c" Sep 11 12:21:33.093140 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:21:33.093122 2566 scope.go:117] "RemoveContainer" containerID="1d39d9d15da2b1855fb6dfae71ae9d6c605e912e35b8b86bcbef9833a084eb1c" Sep 11 12:21:33.093428 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:21:33.093403 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d39d9d15da2b1855fb6dfae71ae9d6c605e912e35b8b86bcbef9833a084eb1c\": container with ID starting with 1d39d9d15da2b1855fb6dfae71ae9d6c605e912e35b8b86bcbef9833a084eb1c not found: ID does not exist" containerID="1d39d9d15da2b1855fb6dfae71ae9d6c605e912e35b8b86bcbef9833a084eb1c" Sep 11 12:21:33.093518 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:21:33.093436 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d39d9d15da2b1855fb6dfae71ae9d6c605e912e35b8b86bcbef9833a084eb1c"} err="failed to get container status \"1d39d9d15da2b1855fb6dfae71ae9d6c605e912e35b8b86bcbef9833a084eb1c\": rpc error: code = NotFound desc = could not find container \"1d39d9d15da2b1855fb6dfae71ae9d6c605e912e35b8b86bcbef9833a084eb1c\": container with ID starting with 1d39d9d15da2b1855fb6dfae71ae9d6c605e912e35b8b86bcbef9833a084eb1c not found: ID does not exist" Sep 11 12:21:33.110428 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:21:33.110400 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f292q"] Sep 11 12:21:33.125652 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:21:33.125623 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f292q"] Sep 11 12:21:34.757944 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:21:34.757908 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16e95dd7-18ae-4528-ab96-79201cf6079c" path="/var/lib/kubelet/pods/16e95dd7-18ae-4528-ab96-79201cf6079c/volumes" Sep 11 12:21:49.032655 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:21:49.032613 2566 scope.go:117] "RemoveContainer" containerID="5ff377c17f78f3ada4d1acf5d0799a31db0b34962d3881bc49969e7d6f1d94dc" Sep 11 12:22:32.863175 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:22:32.863136 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hfxv4"] Sep 11 12:22:32.863774 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:22:32.863574 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16e95dd7-18ae-4528-ab96-79201cf6079c" containerName="manager" Sep 11 12:22:32.863774 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:22:32.863596 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="16e95dd7-18ae-4528-ab96-79201cf6079c" containerName="manager" Sep 11 12:22:32.863774 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:22:32.863702 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="16e95dd7-18ae-4528-ab96-79201cf6079c" containerName="manager" Sep 11 12:22:32.866623 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:22:32.866602 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hfxv4" Sep 11 12:22:32.869856 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:22:32.869836 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-7jt4t\"" Sep 11 12:22:32.879237 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:22:32.879212 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hfxv4"] Sep 11 12:22:33.000661 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:22:33.000628 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pp8k\" (UniqueName: \"kubernetes.io/projected/08dda338-3f0c-4701-8beb-dd35d47e8f2c-kube-api-access-7pp8k\") pod \"kuadrant-operator-controller-manager-5769f99d9f-hfxv4\" (UID: \"08dda338-3f0c-4701-8beb-dd35d47e8f2c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hfxv4" Sep 11 12:22:33.000837 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:22:33.000766 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/08dda338-3f0c-4701-8beb-dd35d47e8f2c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-hfxv4\" (UID: \"08dda338-3f0c-4701-8beb-dd35d47e8f2c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hfxv4" Sep 11 12:22:33.101753 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:22:33.101709 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/08dda338-3f0c-4701-8beb-dd35d47e8f2c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-hfxv4\" (UID: \"08dda338-3f0c-4701-8beb-dd35d47e8f2c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hfxv4" Sep 11 12:22:33.101984 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:22:33.101777 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pp8k\" (UniqueName: \"kubernetes.io/projected/08dda338-3f0c-4701-8beb-dd35d47e8f2c-kube-api-access-7pp8k\") pod \"kuadrant-operator-controller-manager-5769f99d9f-hfxv4\" (UID: \"08dda338-3f0c-4701-8beb-dd35d47e8f2c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hfxv4" Sep 11 12:22:33.102152 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:22:33.102116 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/08dda338-3f0c-4701-8beb-dd35d47e8f2c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-hfxv4\" (UID: \"08dda338-3f0c-4701-8beb-dd35d47e8f2c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hfxv4" Sep 11 12:22:33.111008 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:22:33.110988 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pp8k\" (UniqueName: \"kubernetes.io/projected/08dda338-3f0c-4701-8beb-dd35d47e8f2c-kube-api-access-7pp8k\") pod \"kuadrant-operator-controller-manager-5769f99d9f-hfxv4\" (UID: \"08dda338-3f0c-4701-8beb-dd35d47e8f2c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hfxv4" Sep 11 12:22:33.176996 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:22:33.176902 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hfxv4" Sep 11 12:22:33.469172 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:22:33.469095 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hfxv4"] Sep 11 12:22:33.472572 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:22:33.472546 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08dda338_3f0c_4701_8beb_dd35d47e8f2c.slice/crio-39289969c742022243755dcc055d8169f8aa6248f4bb700dbcdde33bd5bc6706 WatchSource:0}: Error finding container 39289969c742022243755dcc055d8169f8aa6248f4bb700dbcdde33bd5bc6706: Status 404 returned error can't find the container with id 39289969c742022243755dcc055d8169f8aa6248f4bb700dbcdde33bd5bc6706 Sep 11 12:22:34.356400 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:22:34.356362 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hfxv4" event={"ID":"08dda338-3f0c-4701-8beb-dd35d47e8f2c","Type":"ContainerStarted","Data":"1ce82efc498638392b727bbd5cf977991fb094218707c14992c0b28d1bab499b"} Sep 11 12:22:34.356901 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:22:34.356440 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hfxv4" event={"ID":"08dda338-3f0c-4701-8beb-dd35d47e8f2c","Type":"ContainerStarted","Data":"39289969c742022243755dcc055d8169f8aa6248f4bb700dbcdde33bd5bc6706"} Sep 11 12:22:34.356901 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:22:34.356464 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hfxv4" Sep 11 12:22:34.380655 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:22:34.380594 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hfxv4" podStartSLOduration=2.380576004 podStartE2EDuration="2.380576004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:22:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:22:34.378318866 +0000 UTC m=+2086.143292041" watchObservedRunningTime="2026-09-11 12:22:34.380576004 +0000 UTC m=+2086.145549178" Sep 11 12:22:45.363107 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:22:45.363072 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hfxv4" Sep 11 12:22:49.052888 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:22:49.052828 2566 scope.go:117] "RemoveContainer" containerID="b3a0f161f37fa88d79d9b12c541a7a6926d4f12063086dc9659acaa8cca0ca81" Sep 11 12:23:35.035248 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:23:35.035200 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx"] Sep 11 12:23:35.038556 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:23:35.038530 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx" Sep 11 12:23:35.041071 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:23:35.041048 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-e0808799-kserve-self-signed-certs\"" Sep 11 12:23:35.050400 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:23:35.050373 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx"] Sep 11 12:23:35.136804 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:23:35.136763 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83567b07-3d51-4457-822b-f6ef3e786624-tls-certs\") pod \"e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx\" (UID: \"83567b07-3d51-4457-822b-f6ef3e786624\") " pod="llm/e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx" Sep 11 12:23:35.136804 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:23:35.136810 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83567b07-3d51-4457-822b-f6ef3e786624-dshm\") pod \"e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx\" (UID: \"83567b07-3d51-4457-822b-f6ef3e786624\") " pod="llm/e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx" Sep 11 12:23:35.137085 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:23:35.136836 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83567b07-3d51-4457-822b-f6ef3e786624-model-cache\") pod \"e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx\" (UID: \"83567b07-3d51-4457-822b-f6ef3e786624\") " pod="llm/e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx" Sep 11 12:23:35.137085 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:23:35.136966 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wscqd\" (UniqueName: \"kubernetes.io/projected/83567b07-3d51-4457-822b-f6ef3e786624-kube-api-access-wscqd\") pod \"e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx\" (UID: \"83567b07-3d51-4457-822b-f6ef3e786624\") " pod="llm/e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx" Sep 11 12:23:35.137085 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:23:35.137016 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83567b07-3d51-4457-822b-f6ef3e786624-home\") pod \"e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx\" (UID: \"83567b07-3d51-4457-822b-f6ef3e786624\") " pod="llm/e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx" Sep 11 12:23:35.137085 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:23:35.137074 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83567b07-3d51-4457-822b-f6ef3e786624-tmp-dir\") pod \"e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx\" (UID: \"83567b07-3d51-4457-822b-f6ef3e786624\") " pod="llm/e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx" Sep 11 12:23:35.238000 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:23:35.237962 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wscqd\" (UniqueName: \"kubernetes.io/projected/83567b07-3d51-4457-822b-f6ef3e786624-kube-api-access-wscqd\") pod \"e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx\" (UID: \"83567b07-3d51-4457-822b-f6ef3e786624\") " pod="llm/e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx" Sep 11 12:23:35.238200 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:23:35.238007 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83567b07-3d51-4457-822b-f6ef3e786624-home\") pod \"e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx\" (UID: \"83567b07-3d51-4457-822b-f6ef3e786624\") " pod="llm/e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx" Sep 11 12:23:35.238200 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:23:35.238034 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83567b07-3d51-4457-822b-f6ef3e786624-tmp-dir\") pod \"e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx\" (UID: \"83567b07-3d51-4457-822b-f6ef3e786624\") " pod="llm/e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx" Sep 11 12:23:35.238200 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:23:35.238063 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83567b07-3d51-4457-822b-f6ef3e786624-tls-certs\") pod \"e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx\" (UID: \"83567b07-3d51-4457-822b-f6ef3e786624\") " pod="llm/e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx" Sep 11 12:23:35.238200 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:23:35.238094 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83567b07-3d51-4457-822b-f6ef3e786624-dshm\") pod \"e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx\" (UID: \"83567b07-3d51-4457-822b-f6ef3e786624\") " pod="llm/e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx" Sep 11 12:23:35.238200 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:23:35.238125 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83567b07-3d51-4457-822b-f6ef3e786624-model-cache\") pod \"e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx\" (UID: \"83567b07-3d51-4457-822b-f6ef3e786624\") " pod="llm/e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx" Sep 11 12:23:35.238472 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:23:35.238450 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83567b07-3d51-4457-822b-f6ef3e786624-tmp-dir\") pod \"e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx\" (UID: \"83567b07-3d51-4457-822b-f6ef3e786624\") " pod="llm/e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx" Sep 11 12:23:35.238530 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:23:35.238459 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83567b07-3d51-4457-822b-f6ef3e786624-home\") pod \"e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx\" (UID: \"83567b07-3d51-4457-822b-f6ef3e786624\") " pod="llm/e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx" Sep 11 12:23:35.238589 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:23:35.238539 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83567b07-3d51-4457-822b-f6ef3e786624-model-cache\") pod \"e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx\" (UID: \"83567b07-3d51-4457-822b-f6ef3e786624\") " pod="llm/e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx" Sep 11 12:23:35.240371 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:23:35.240347 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83567b07-3d51-4457-822b-f6ef3e786624-dshm\") pod \"e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx\" (UID: \"83567b07-3d51-4457-822b-f6ef3e786624\") " pod="llm/e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx" Sep 11 12:23:35.240627 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:23:35.240604 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83567b07-3d51-4457-822b-f6ef3e786624-tls-certs\") pod \"e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx\" (UID: \"83567b07-3d51-4457-822b-f6ef3e786624\") " pod="llm/e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx" Sep 11 12:23:35.247462 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:23:35.247435 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wscqd\" (UniqueName: \"kubernetes.io/projected/83567b07-3d51-4457-822b-f6ef3e786624-kube-api-access-wscqd\") pod \"e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx\" (UID: \"83567b07-3d51-4457-822b-f6ef3e786624\") " pod="llm/e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx" Sep 11 12:23:35.348429 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:23:35.348387 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx" Sep 11 12:23:35.621581 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:23:35.621557 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx"] Sep 11 12:23:35.627195 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:23:35.627163 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod83567b07_3d51_4457_822b_f6ef3e786624.slice/crio-fa8ca3846f5e8f0bf91e2154d84b4d273128d84e00785af3664b567250becb00 WatchSource:0}: Error finding container fa8ca3846f5e8f0bf91e2154d84b4d273128d84e00785af3664b567250becb00: Status 404 returned error can't find the container with id fa8ca3846f5e8f0bf91e2154d84b4d273128d84e00785af3664b567250becb00 Sep 11 12:23:36.630087 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:23:36.630047 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx" event={"ID":"83567b07-3d51-4457-822b-f6ef3e786624","Type":"ContainerStarted","Data":"5c55d5a54b90c9c3f63d27a6cd4e415384904f5ab32c97562ab11c633993c87c"} Sep 11 12:23:36.630516 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:23:36.630095 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx" event={"ID":"83567b07-3d51-4457-822b-f6ef3e786624","Type":"ContainerStarted","Data":"fa8ca3846f5e8f0bf91e2154d84b4d273128d84e00785af3664b567250becb00"} Sep 11 12:23:36.650932 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:23:36.650861 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx" podStartSLOduration=1.650844609 podStartE2EDuration="1.650844609s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:23:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:23:36.649289514 +0000 UTC m=+2148.414262681" watchObservedRunningTime="2026-09-11 12:23:36.650844609 +0000 UTC m=+2148.415817784" Sep 11 12:23:45.348760 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:23:45.348663 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx" Sep 11 12:23:45.348760 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:23:45.348719 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx" Sep 11 12:23:45.361608 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:23:45.361579 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx" Sep 11 12:23:45.683873 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:23:45.683778 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx" Sep 11 12:24:13.329820 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:13.329784 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx"] Sep 11 12:24:28.348981 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:28.348937 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx" podUID="83567b07-3d51-4457-822b-f6ef3e786624" containerName="main" containerID="cri-o://5c55d5a54b90c9c3f63d27a6cd4e415384904f5ab32c97562ab11c633993c87c" gracePeriod=45 Sep 11 12:24:28.590940 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:28.590909 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx" Sep 11 12:24:28.696368 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:28.696283 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83567b07-3d51-4457-822b-f6ef3e786624-dshm\") pod \"83567b07-3d51-4457-822b-f6ef3e786624\" (UID: \"83567b07-3d51-4457-822b-f6ef3e786624\") " Sep 11 12:24:28.696368 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:28.696346 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83567b07-3d51-4457-822b-f6ef3e786624-home\") pod \"83567b07-3d51-4457-822b-f6ef3e786624\" (UID: \"83567b07-3d51-4457-822b-f6ef3e786624\") " Sep 11 12:24:28.696586 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:28.696382 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wscqd\" (UniqueName: \"kubernetes.io/projected/83567b07-3d51-4457-822b-f6ef3e786624-kube-api-access-wscqd\") pod \"83567b07-3d51-4457-822b-f6ef3e786624\" (UID: \"83567b07-3d51-4457-822b-f6ef3e786624\") " Sep 11 12:24:28.696586 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:28.696427 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83567b07-3d51-4457-822b-f6ef3e786624-model-cache\") pod \"83567b07-3d51-4457-822b-f6ef3e786624\" (UID: \"83567b07-3d51-4457-822b-f6ef3e786624\") " Sep 11 12:24:28.696586 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:28.696452 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83567b07-3d51-4457-822b-f6ef3e786624-tmp-dir\") pod \"83567b07-3d51-4457-822b-f6ef3e786624\" (UID: \"83567b07-3d51-4457-822b-f6ef3e786624\") " Sep 11 12:24:28.696586 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:28.696495 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83567b07-3d51-4457-822b-f6ef3e786624-tls-certs\") pod \"83567b07-3d51-4457-822b-f6ef3e786624\" (UID: \"83567b07-3d51-4457-822b-f6ef3e786624\") " Sep 11 12:24:28.696586 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:28.696562 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83567b07-3d51-4457-822b-f6ef3e786624-home" (OuterVolumeSpecName: "home") pod "83567b07-3d51-4457-822b-f6ef3e786624" (UID: "83567b07-3d51-4457-822b-f6ef3e786624"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:24:28.696858 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:28.696660 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83567b07-3d51-4457-822b-f6ef3e786624-model-cache" (OuterVolumeSpecName: "model-cache") pod "83567b07-3d51-4457-822b-f6ef3e786624" (UID: "83567b07-3d51-4457-822b-f6ef3e786624"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:24:28.696858 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:28.696788 2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83567b07-3d51-4457-822b-f6ef3e786624-home\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:24:28.696858 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:28.696807 2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83567b07-3d51-4457-822b-f6ef3e786624-model-cache\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:24:28.696858 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:28.696803 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83567b07-3d51-4457-822b-f6ef3e786624-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "83567b07-3d51-4457-822b-f6ef3e786624" (UID: "83567b07-3d51-4457-822b-f6ef3e786624"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:24:28.698726 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:28.698698 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83567b07-3d51-4457-822b-f6ef3e786624-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "83567b07-3d51-4457-822b-f6ef3e786624" (UID: "83567b07-3d51-4457-822b-f6ef3e786624"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 12:24:28.698853 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:28.698728 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83567b07-3d51-4457-822b-f6ef3e786624-kube-api-access-wscqd" (OuterVolumeSpecName: "kube-api-access-wscqd") pod "83567b07-3d51-4457-822b-f6ef3e786624" (UID: "83567b07-3d51-4457-822b-f6ef3e786624"). InnerVolumeSpecName "kube-api-access-wscqd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:24:28.698939 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:28.698853 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83567b07-3d51-4457-822b-f6ef3e786624-dshm" (OuterVolumeSpecName: "dshm") pod "83567b07-3d51-4457-822b-f6ef3e786624" (UID: "83567b07-3d51-4457-822b-f6ef3e786624"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 12:24:28.797301 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:28.797267 2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83567b07-3d51-4457-822b-f6ef3e786624-tmp-dir\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:24:28.797301 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:28.797300 2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83567b07-3d51-4457-822b-f6ef3e786624-tls-certs\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:24:28.797526 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:28.797313 2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83567b07-3d51-4457-822b-f6ef3e786624-dshm\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:24:28.797526 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:28.797321 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wscqd\" (UniqueName: \"kubernetes.io/projected/83567b07-3d51-4457-822b-f6ef3e786624-kube-api-access-wscqd\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:24:28.871400 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:28.871361 2566 generic.go:356] "Generic (PLEG): container finished" podID="83567b07-3d51-4457-822b-f6ef3e786624" containerID="5c55d5a54b90c9c3f63d27a6cd4e415384904f5ab32c97562ab11c633993c87c" exitCode=0 Sep 11 12:24:28.871585 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:28.871418 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx" Sep 11 12:24:28.871585 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:28.871454 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx" event={"ID":"83567b07-3d51-4457-822b-f6ef3e786624","Type":"ContainerDied","Data":"5c55d5a54b90c9c3f63d27a6cd4e415384904f5ab32c97562ab11c633993c87c"} Sep 11 12:24:28.871585 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:28.871508 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx" event={"ID":"83567b07-3d51-4457-822b-f6ef3e786624","Type":"ContainerDied","Data":"fa8ca3846f5e8f0bf91e2154d84b4d273128d84e00785af3664b567250becb00"} Sep 11 12:24:28.871585 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:28.871532 2566 scope.go:117] "RemoveContainer" containerID="5c55d5a54b90c9c3f63d27a6cd4e415384904f5ab32c97562ab11c633993c87c" Sep 11 12:24:28.879893 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:28.879859 2566 scope.go:117] "RemoveContainer" containerID="5c55d5a54b90c9c3f63d27a6cd4e415384904f5ab32c97562ab11c633993c87c" Sep 11 12:24:28.880169 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:24:28.880148 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c55d5a54b90c9c3f63d27a6cd4e415384904f5ab32c97562ab11c633993c87c\": container with ID starting with 5c55d5a54b90c9c3f63d27a6cd4e415384904f5ab32c97562ab11c633993c87c not found: ID does not exist" containerID="5c55d5a54b90c9c3f63d27a6cd4e415384904f5ab32c97562ab11c633993c87c" Sep 11 12:24:28.880291 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:28.880179 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c55d5a54b90c9c3f63d27a6cd4e415384904f5ab32c97562ab11c633993c87c"} err="failed to get container status \"5c55d5a54b90c9c3f63d27a6cd4e415384904f5ab32c97562ab11c633993c87c\": rpc error: code = NotFound desc = could not find container \"5c55d5a54b90c9c3f63d27a6cd4e415384904f5ab32c97562ab11c633993c87c\": container with ID starting with 5c55d5a54b90c9c3f63d27a6cd4e415384904f5ab32c97562ab11c633993c87c not found: ID does not exist" Sep 11 12:24:28.890241 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:28.890219 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx"] Sep 11 12:24:28.896651 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:28.896631 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-e0808799-kserve-7665d874cf-pzbgx"] Sep 11 12:24:30.757825 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:30.757778 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83567b07-3d51-4457-822b-f6ef3e786624" path="/var/lib/kubelet/pods/83567b07-3d51-4457-822b-f6ef3e786624/volumes" Sep 11 12:24:40.333003 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:40.332962 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7b8f67d746-q85bx"] Sep 11 12:24:40.333368 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:40.333279 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83567b07-3d51-4457-822b-f6ef3e786624" containerName="main" Sep 11 12:24:40.333368 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:40.333291 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="83567b07-3d51-4457-822b-f6ef3e786624" containerName="main" Sep 11 12:24:40.333368 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:40.333357 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="83567b07-3d51-4457-822b-f6ef3e786624" containerName="main" Sep 11 12:24:40.337439 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:40.337421 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7b8f67d746-q85bx" Sep 11 12:24:40.345656 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:40.345626 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7b8f67d746-q85bx"] Sep 11 12:24:40.401807 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:40.401775 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81d104a1-325c-4266-901e-6b2f37fcd04b-maas-api-tls\") pod \"maas-api-7b8f67d746-q85bx\" (UID: \"81d104a1-325c-4266-901e-6b2f37fcd04b\") " pod="odh-ai-gateway-infra/maas-api-7b8f67d746-q85bx" Sep 11 12:24:40.401807 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:40.401810 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81d104a1-325c-4266-901e-6b2f37fcd04b-metrics-certs\") pod \"maas-api-7b8f67d746-q85bx\" (UID: \"81d104a1-325c-4266-901e-6b2f37fcd04b\") " pod="odh-ai-gateway-infra/maas-api-7b8f67d746-q85bx" Sep 11 12:24:40.402061 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:40.401837 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81d104a1-325c-4266-901e-6b2f37fcd04b-openshift-service-ca\") pod \"maas-api-7b8f67d746-q85bx\" (UID: \"81d104a1-325c-4266-901e-6b2f37fcd04b\") " pod="odh-ai-gateway-infra/maas-api-7b8f67d746-q85bx" Sep 11 12:24:40.402061 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:40.401973 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xjnb\" (UniqueName: \"kubernetes.io/projected/81d104a1-325c-4266-901e-6b2f37fcd04b-kube-api-access-5xjnb\") pod \"maas-api-7b8f67d746-q85bx\" (UID: \"81d104a1-325c-4266-901e-6b2f37fcd04b\") " pod="odh-ai-gateway-infra/maas-api-7b8f67d746-q85bx" Sep 11 12:24:40.502455 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:40.502410 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xjnb\" (UniqueName: \"kubernetes.io/projected/81d104a1-325c-4266-901e-6b2f37fcd04b-kube-api-access-5xjnb\") pod \"maas-api-7b8f67d746-q85bx\" (UID: \"81d104a1-325c-4266-901e-6b2f37fcd04b\") " pod="odh-ai-gateway-infra/maas-api-7b8f67d746-q85bx" Sep 11 12:24:40.502657 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:40.502479 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81d104a1-325c-4266-901e-6b2f37fcd04b-maas-api-tls\") pod \"maas-api-7b8f67d746-q85bx\" (UID: \"81d104a1-325c-4266-901e-6b2f37fcd04b\") " pod="odh-ai-gateway-infra/maas-api-7b8f67d746-q85bx" Sep 11 12:24:40.502657 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:40.502501 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81d104a1-325c-4266-901e-6b2f37fcd04b-metrics-certs\") pod \"maas-api-7b8f67d746-q85bx\" (UID: \"81d104a1-325c-4266-901e-6b2f37fcd04b\") " pod="odh-ai-gateway-infra/maas-api-7b8f67d746-q85bx" Sep 11 12:24:40.502657 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:40.502518 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81d104a1-325c-4266-901e-6b2f37fcd04b-openshift-service-ca\") pod \"maas-api-7b8f67d746-q85bx\" (UID: \"81d104a1-325c-4266-901e-6b2f37fcd04b\") " pod="odh-ai-gateway-infra/maas-api-7b8f67d746-q85bx" Sep 11 12:24:40.503367 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:40.503337 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81d104a1-325c-4266-901e-6b2f37fcd04b-openshift-service-ca\") pod \"maas-api-7b8f67d746-q85bx\" (UID: \"81d104a1-325c-4266-901e-6b2f37fcd04b\") " pod="odh-ai-gateway-infra/maas-api-7b8f67d746-q85bx" Sep 11 12:24:40.505004 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:40.504979 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81d104a1-325c-4266-901e-6b2f37fcd04b-maas-api-tls\") pod \"maas-api-7b8f67d746-q85bx\" (UID: \"81d104a1-325c-4266-901e-6b2f37fcd04b\") " pod="odh-ai-gateway-infra/maas-api-7b8f67d746-q85bx" Sep 11 12:24:40.505380 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:40.505359 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81d104a1-325c-4266-901e-6b2f37fcd04b-metrics-certs\") pod \"maas-api-7b8f67d746-q85bx\" (UID: \"81d104a1-325c-4266-901e-6b2f37fcd04b\") " pod="odh-ai-gateway-infra/maas-api-7b8f67d746-q85bx" Sep 11 12:24:40.512048 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:40.512013 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xjnb\" (UniqueName: \"kubernetes.io/projected/81d104a1-325c-4266-901e-6b2f37fcd04b-kube-api-access-5xjnb\") pod \"maas-api-7b8f67d746-q85bx\" (UID: \"81d104a1-325c-4266-901e-6b2f37fcd04b\") " pod="odh-ai-gateway-infra/maas-api-7b8f67d746-q85bx" Sep 11 12:24:40.648555 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:40.648466 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7b8f67d746-q85bx" Sep 11 12:24:40.924518 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:40.924487 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7b8f67d746-q85bx"] Sep 11 12:24:40.927940 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:24:40.927905 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81d104a1_325c_4266_901e_6b2f37fcd04b.slice/crio-5591d8fcdb871a26090d29cd2ec0230fa8b55e15a48b405c0aa49c795a21f1d1 WatchSource:0}: Error finding container 5591d8fcdb871a26090d29cd2ec0230fa8b55e15a48b405c0aa49c795a21f1d1: Status 404 returned error can't find the container with id 5591d8fcdb871a26090d29cd2ec0230fa8b55e15a48b405c0aa49c795a21f1d1 Sep 11 12:24:41.933899 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:41.933844 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7b8f67d746-q85bx" event={"ID":"81d104a1-325c-4266-901e-6b2f37fcd04b","Type":"ContainerStarted","Data":"5591d8fcdb871a26090d29cd2ec0230fa8b55e15a48b405c0aa49c795a21f1d1"} Sep 11 12:24:42.940220 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:42.940181 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7b8f67d746-q85bx" event={"ID":"81d104a1-325c-4266-901e-6b2f37fcd04b","Type":"ContainerStarted","Data":"13da857591d1f1f9ca35d42be423a7fc47fddb67170d9c8011ca23f7fd83dbe7"} Sep 11 12:24:42.940585 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:42.940253 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7b8f67d746-q85bx" Sep 11 12:24:42.959250 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:42.959207 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7b8f67d746-q85bx" podStartSLOduration=1.930565396 podStartE2EDuration="2.959189137s" totalImagesPullingTime="1.028623741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:24:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:24:42.957166641 +0000 UTC m=+2214.722139816" watchObservedRunningTime="2026-09-11 12:24:42.959189137 +0000 UTC m=+2214.724162314" Sep 11 12:24:48.951504 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:48.951474 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7b8f67d746-q85bx" Sep 11 12:24:49.057587 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:49.057549 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-669df8b46-wcvgj"] Sep 11 12:24:49.057802 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:49.057758 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-669df8b46-wcvgj" podUID="f09434f9-9518-46b1-9311-03bfa6265118" containerName="maas-api" containerID="cri-o://0f7456d79ccb6734312b7a6bbae80d76985e17e9e8c17f4441668118436a7cb7" gracePeriod=30 Sep 11 12:24:49.252027 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:49.251926 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-669df8b46-wcvgj" podUID="f09434f9-9518-46b1-9311-03bfa6265118" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.26:8443/health\": dial tcp 10.132.0.26:8443: connect: connection refused" Sep 11 12:24:50.295091 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:50.295067 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-669df8b46-wcvgj" Sep 11 12:24:50.391207 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:50.391165 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f09434f9-9518-46b1-9311-03bfa6265118-openshift-service-ca\") pod \"f09434f9-9518-46b1-9311-03bfa6265118\" (UID: \"f09434f9-9518-46b1-9311-03bfa6265118\") " Sep 11 12:24:50.391396 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:50.391220 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4jk7n\" (UniqueName: \"kubernetes.io/projected/f09434f9-9518-46b1-9311-03bfa6265118-kube-api-access-4jk7n\") pod \"f09434f9-9518-46b1-9311-03bfa6265118\" (UID: \"f09434f9-9518-46b1-9311-03bfa6265118\") " Sep 11 12:24:50.391396 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:50.391240 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f09434f9-9518-46b1-9311-03bfa6265118-maas-api-tls\") pod \"f09434f9-9518-46b1-9311-03bfa6265118\" (UID: \"f09434f9-9518-46b1-9311-03bfa6265118\") " Sep 11 12:24:50.391396 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:50.391288 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f09434f9-9518-46b1-9311-03bfa6265118-metrics-certs\") pod \"f09434f9-9518-46b1-9311-03bfa6265118\" (UID: \"f09434f9-9518-46b1-9311-03bfa6265118\") " Sep 11 12:24:50.391613 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:50.391588 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f09434f9-9518-46b1-9311-03bfa6265118-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f09434f9-9518-46b1-9311-03bfa6265118" (UID: "f09434f9-9518-46b1-9311-03bfa6265118"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 12:24:50.393433 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:50.393404 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f09434f9-9518-46b1-9311-03bfa6265118-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f09434f9-9518-46b1-9311-03bfa6265118" (UID: "f09434f9-9518-46b1-9311-03bfa6265118"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 12:24:50.393559 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:50.393459 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f09434f9-9518-46b1-9311-03bfa6265118-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f09434f9-9518-46b1-9311-03bfa6265118" (UID: "f09434f9-9518-46b1-9311-03bfa6265118"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 12:24:50.393559 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:50.393460 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f09434f9-9518-46b1-9311-03bfa6265118-kube-api-access-4jk7n" (OuterVolumeSpecName: "kube-api-access-4jk7n") pod "f09434f9-9518-46b1-9311-03bfa6265118" (UID: "f09434f9-9518-46b1-9311-03bfa6265118"). InnerVolumeSpecName "kube-api-access-4jk7n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 12:24:50.492349 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:50.492318 2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f09434f9-9518-46b1-9311-03bfa6265118-openshift-service-ca\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:24:50.492349 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:50.492346 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4jk7n\" (UniqueName: \"kubernetes.io/projected/f09434f9-9518-46b1-9311-03bfa6265118-kube-api-access-4jk7n\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:24:50.492544 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:50.492358 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f09434f9-9518-46b1-9311-03bfa6265118-maas-api-tls\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:24:50.492544 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:50.492368 2566 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f09434f9-9518-46b1-9311-03bfa6265118-metrics-certs\") on node \"ip-10-0-132-170.ec2.internal\" DevicePath \"\"" Sep 11 12:24:50.977526 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:50.977492 2566 generic.go:356] "Generic (PLEG): container finished" podID="f09434f9-9518-46b1-9311-03bfa6265118" containerID="0f7456d79ccb6734312b7a6bbae80d76985e17e9e8c17f4441668118436a7cb7" exitCode=0 Sep 11 12:24:50.977696 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:50.977538 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-669df8b46-wcvgj" Sep 11 12:24:50.977696 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:50.977583 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-669df8b46-wcvgj" event={"ID":"f09434f9-9518-46b1-9311-03bfa6265118","Type":"ContainerDied","Data":"0f7456d79ccb6734312b7a6bbae80d76985e17e9e8c17f4441668118436a7cb7"} Sep 11 12:24:50.977696 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:50.977636 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-669df8b46-wcvgj" event={"ID":"f09434f9-9518-46b1-9311-03bfa6265118","Type":"ContainerDied","Data":"cddbb1195c6a15fdaf04879c56e266aa8d713a8134bff39edd8075267fdd5c2d"} Sep 11 12:24:50.977696 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:50.977661 2566 scope.go:117] "RemoveContainer" containerID="0f7456d79ccb6734312b7a6bbae80d76985e17e9e8c17f4441668118436a7cb7" Sep 11 12:24:50.986325 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:50.986308 2566 scope.go:117] "RemoveContainer" containerID="0f7456d79ccb6734312b7a6bbae80d76985e17e9e8c17f4441668118436a7cb7" Sep 11 12:24:50.986593 ip-10-0-132-170 kubenswrapper[2566]: E0911 12:24:50.986573 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f7456d79ccb6734312b7a6bbae80d76985e17e9e8c17f4441668118436a7cb7\": container with ID starting with 0f7456d79ccb6734312b7a6bbae80d76985e17e9e8c17f4441668118436a7cb7 not found: ID does not exist" containerID="0f7456d79ccb6734312b7a6bbae80d76985e17e9e8c17f4441668118436a7cb7" Sep 11 12:24:50.986636 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:50.986601 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f7456d79ccb6734312b7a6bbae80d76985e17e9e8c17f4441668118436a7cb7"} err="failed to get container status \"0f7456d79ccb6734312b7a6bbae80d76985e17e9e8c17f4441668118436a7cb7\": rpc error: code = NotFound desc = could not find container \"0f7456d79ccb6734312b7a6bbae80d76985e17e9e8c17f4441668118436a7cb7\": container with ID starting with 0f7456d79ccb6734312b7a6bbae80d76985e17e9e8c17f4441668118436a7cb7 not found: ID does not exist" Sep 11 12:24:50.996880 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:50.996841 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-669df8b46-wcvgj"] Sep 11 12:24:51.000511 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:51.000476 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-669df8b46-wcvgj"] Sep 11 12:24:52.758885 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:24:52.758846 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f09434f9-9518-46b1-9311-03bfa6265118" path="/var/lib/kubelet/pods/f09434f9-9518-46b1-9311-03bfa6265118/volumes" Sep 11 12:25:09.067464 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:09.067428 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-c4fd66f5c-lnlwl_df5f488c-1c66-462d-a1c3-b8abe43dd74f/manager/0.log" Sep 11 12:25:09.717852 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:09.717821 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fb8846894-zcf9q_77edd868-8a77-4e81-8bf9-301419e19c8d/copy-manifests/0.log" Sep 11 12:25:09.732387 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:09.732358 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fb8846894-zcf9q_77edd868-8a77-4e81-8bf9-301419e19c8d/manager/0.log" Sep 11 12:25:10.240152 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:10.240123 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-7b8f67d746-q85bx_81d104a1-325c-4266-901e-6b2f37fcd04b/maas-api/0.log" Sep 11 12:25:10.479204 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:10.479172 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29818815-ww8b8_1c476fe9-7d91-4ae5-bf84-72180ba00705/cleanup/0.log" Sep 11 12:25:11.717209 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:11.717132 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-gmx4v_413baf89-6e9c-4338-9ee3-b09da8291801/manager/0.log" Sep 11 12:25:11.833558 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:11.833528 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-8dbr8_835ff850-43f8-495f-813f-4d2950a8749e/kuadrant-console-plugin/0.log" Sep 11 12:25:11.961154 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:11.961124 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-hfxv4_08dda338-3f0c-4701-8beb-dd35d47e8f2c/manager/0.log" Sep 11 12:25:12.546483 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:12.546454 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5_63b1165a-60a3-4edc-b78d-af87251b58ec/istio-proxy/0.log" Sep 11 12:25:13.056885 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:13.056850 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-7v444_c888eade-3e00-4c36-b236-6c86b58d33bc/istio-proxy/1.log" Sep 11 12:25:13.314168 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:13.314097 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-pq4hx_e55f3a24-f1c7-4556-a5eb-8a9ba6e89fe1/payload-processing/0.log" Sep 11 12:25:14.101643 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:14.101611 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-pwzdw_581ca9c6-c1f4-474c-82a1-51bf6ddd6822/main/0.log" Sep 11 12:25:14.491799 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:14.491716 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-gjc9l_1e88b32b-bc59-44f4-a542-41567be1ada7/main/0.log" Sep 11 12:25:23.206466 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:23.206439 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-tfx78_2dc78d6d-cf10-4087-a38b-0ded1057f222/global-pull-secret-syncer/0.log" Sep 11 12:25:23.295574 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:23.295543 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-n9pc8_f90862ea-2909-4701-ab56-fe2d3fbb83f8/konnectivity-agent/0.log" Sep 11 12:25:23.357565 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:23.357533 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-170.ec2.internal_579298bbec0b9e8e74fae2628d563376/haproxy/0.log" Sep 11 12:25:27.662083 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:27.662054 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-gmx4v_413baf89-6e9c-4338-9ee3-b09da8291801/manager/0.log" Sep 11 12:25:27.690999 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:27.690969 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-8dbr8_835ff850-43f8-495f-813f-4d2950a8749e/kuadrant-console-plugin/0.log" Sep 11 12:25:27.731665 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:27.731639 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-hfxv4_08dda338-3f0c-4701-8beb-dd35d47e8f2c/manager/0.log" Sep 11 12:25:29.955597 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:29.955567 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-48knj_45cf0550-9900-445d-8813-ff3881e3fec9/node-exporter/0.log" Sep 11 12:25:29.983733 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:29.983701 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-48knj_45cf0550-9900-445d-8813-ff3881e3fec9/kube-rbac-proxy/0.log" Sep 11 12:25:30.015759 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:30.015732 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-48knj_45cf0550-9900-445d-8813-ff3881e3fec9/init-textfile/0.log" Sep 11 12:25:32.988638 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:32.988592 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vdhzr/perf-node-gather-daemonset-d2k5f"] Sep 11 12:25:32.989038 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:32.988951 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f09434f9-9518-46b1-9311-03bfa6265118" containerName="maas-api" Sep 11 12:25:32.989038 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:32.988964 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f09434f9-9518-46b1-9311-03bfa6265118" containerName="maas-api" Sep 11 12:25:32.989038 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:32.989029 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f09434f9-9518-46b1-9311-03bfa6265118" containerName="maas-api" Sep 11 12:25:32.992085 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:32.992063 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-d2k5f" Sep 11 12:25:32.994530 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:32.994504 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-vdhzr\"/\"default-dockercfg-m29sz\"" Sep 11 12:25:32.994632 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:32.994561 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vdhzr\"/\"openshift-service-ca.crt\"" Sep 11 12:25:32.994632 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:32.994568 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vdhzr\"/\"kube-root-ca.crt\"" Sep 11 12:25:33.000639 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:33.000607 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vdhzr/perf-node-gather-daemonset-d2k5f"] Sep 11 12:25:33.058979 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:33.058945 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/65c04709-418e-498d-b3e7-ac53869da8ca-lib-modules\") pod \"perf-node-gather-daemonset-d2k5f\" (UID: \"65c04709-418e-498d-b3e7-ac53869da8ca\") " pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-d2k5f" Sep 11 12:25:33.059160 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:33.059016 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/65c04709-418e-498d-b3e7-ac53869da8ca-podres\") pod \"perf-node-gather-daemonset-d2k5f\" (UID: \"65c04709-418e-498d-b3e7-ac53869da8ca\") " pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-d2k5f" Sep 11 12:25:33.059160 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:33.059062 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpq65\" (UniqueName: \"kubernetes.io/projected/65c04709-418e-498d-b3e7-ac53869da8ca-kube-api-access-vpq65\") pod \"perf-node-gather-daemonset-d2k5f\" (UID: \"65c04709-418e-498d-b3e7-ac53869da8ca\") " pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-d2k5f" Sep 11 12:25:33.059160 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:33.059109 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/65c04709-418e-498d-b3e7-ac53869da8ca-proc\") pod \"perf-node-gather-daemonset-d2k5f\" (UID: \"65c04709-418e-498d-b3e7-ac53869da8ca\") " pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-d2k5f" Sep 11 12:25:33.059312 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:33.059167 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/65c04709-418e-498d-b3e7-ac53869da8ca-sys\") pod \"perf-node-gather-daemonset-d2k5f\" (UID: \"65c04709-418e-498d-b3e7-ac53869da8ca\") " pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-d2k5f" Sep 11 12:25:33.160509 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:33.160465 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpq65\" (UniqueName: \"kubernetes.io/projected/65c04709-418e-498d-b3e7-ac53869da8ca-kube-api-access-vpq65\") pod \"perf-node-gather-daemonset-d2k5f\" (UID: \"65c04709-418e-498d-b3e7-ac53869da8ca\") " pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-d2k5f" Sep 11 12:25:33.160688 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:33.160538 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/65c04709-418e-498d-b3e7-ac53869da8ca-proc\") pod \"perf-node-gather-daemonset-d2k5f\" (UID: \"65c04709-418e-498d-b3e7-ac53869da8ca\") " pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-d2k5f" Sep 11 12:25:33.160688 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:33.160557 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/65c04709-418e-498d-b3e7-ac53869da8ca-sys\") pod \"perf-node-gather-daemonset-d2k5f\" (UID: \"65c04709-418e-498d-b3e7-ac53869da8ca\") " pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-d2k5f" Sep 11 12:25:33.160688 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:33.160573 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/65c04709-418e-498d-b3e7-ac53869da8ca-lib-modules\") pod \"perf-node-gather-daemonset-d2k5f\" (UID: \"65c04709-418e-498d-b3e7-ac53869da8ca\") " pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-d2k5f" Sep 11 12:25:33.160688 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:33.160672 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/65c04709-418e-498d-b3e7-ac53869da8ca-sys\") pod \"perf-node-gather-daemonset-d2k5f\" (UID: \"65c04709-418e-498d-b3e7-ac53869da8ca\") " pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-d2k5f" Sep 11 12:25:33.160858 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:33.160698 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/65c04709-418e-498d-b3e7-ac53869da8ca-podres\") pod \"perf-node-gather-daemonset-d2k5f\" (UID: \"65c04709-418e-498d-b3e7-ac53869da8ca\") " pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-d2k5f" Sep 11 12:25:33.160858 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:33.160713 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/65c04709-418e-498d-b3e7-ac53869da8ca-lib-modules\") pod \"perf-node-gather-daemonset-d2k5f\" (UID: \"65c04709-418e-498d-b3e7-ac53869da8ca\") " pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-d2k5f" Sep 11 12:25:33.160858 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:33.160672 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/65c04709-418e-498d-b3e7-ac53869da8ca-proc\") pod \"perf-node-gather-daemonset-d2k5f\" (UID: \"65c04709-418e-498d-b3e7-ac53869da8ca\") " pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-d2k5f" Sep 11 12:25:33.160858 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:33.160786 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/65c04709-418e-498d-b3e7-ac53869da8ca-podres\") pod \"perf-node-gather-daemonset-d2k5f\" (UID: \"65c04709-418e-498d-b3e7-ac53869da8ca\") " pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-d2k5f" Sep 11 12:25:33.169040 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:33.169018 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpq65\" (UniqueName: \"kubernetes.io/projected/65c04709-418e-498d-b3e7-ac53869da8ca-kube-api-access-vpq65\") pod \"perf-node-gather-daemonset-d2k5f\" (UID: \"65c04709-418e-498d-b3e7-ac53869da8ca\") " pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-d2k5f" Sep 11 12:25:33.302971 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:33.302890 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-d2k5f" Sep 11 12:25:33.573240 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:33.573210 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vdhzr/perf-node-gather-daemonset-d2k5f"] Sep 11 12:25:33.575713 ip-10-0-132-170 kubenswrapper[2566]: W0911 12:25:33.575683 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod65c04709_418e_498d_b3e7_ac53869da8ca.slice/crio-dc3205d349c0230b245785bb6b6669106c2bef7576b3bdb07c223a44781f2163 WatchSource:0}: Error finding container dc3205d349c0230b245785bb6b6669106c2bef7576b3bdb07c223a44781f2163: Status 404 returned error can't find the container with id dc3205d349c0230b245785bb6b6669106c2bef7576b3bdb07c223a44781f2163 Sep 11 12:25:34.171789 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:34.171750 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-d2k5f" event={"ID":"65c04709-418e-498d-b3e7-ac53869da8ca","Type":"ContainerStarted","Data":"2ab2dee6238b01ba63e4c5cdb793045b4f6cff03b62b56310e254aaf1ceb5b7e"} Sep 11 12:25:34.172184 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:34.171812 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-d2k5f" event={"ID":"65c04709-418e-498d-b3e7-ac53869da8ca","Type":"ContainerStarted","Data":"dc3205d349c0230b245785bb6b6669106c2bef7576b3bdb07c223a44781f2163"} Sep 11 12:25:34.172184 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:34.171936 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-d2k5f" Sep 11 12:25:34.189666 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:34.189617 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-d2k5f" podStartSLOduration=2.189601344 podStartE2EDuration="2.189601344s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 12:25:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:25:34.188888239 +0000 UTC m=+2265.953861411" watchObservedRunningTime="2026-09-11 12:25:34.189601344 +0000 UTC m=+2265.954574518" Sep 11 12:25:35.348801 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:35.348771 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9f582_6673d6af-e1b2-443b-83c2-7dc7b653b9d0/dns/0.log" Sep 11 12:25:35.371004 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:35.370962 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9f582_6673d6af-e1b2-443b-83c2-7dc7b653b9d0/kube-rbac-proxy/0.log" Sep 11 12:25:35.535061 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:35.535030 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rz4hn_5bf5658e-69eb-4ac8-9359-4653f8fd588f/dns-node-resolver/0.log" Sep 11 12:25:36.120939 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:36.120905 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2sjcg_e0202693-9deb-4f3b-8d1f-3eec2f3f03d6/node-ca/0.log" Sep 11 12:25:37.162034 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:37.162001 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5_63b1165a-60a3-4edc-b78d-af87251b58ec/istio-proxy/0.log" Sep 11 12:25:37.668331 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:37.668277 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-7v444_c888eade-3e00-4c36-b236-6c86b58d33bc/istio-proxy/0.log" Sep 11 12:25:37.675974 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:37.675951 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-7v444_c888eade-3e00-4c36-b236-6c86b58d33bc/istio-proxy/1.log" Sep 11 12:25:37.763795 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:37.763767 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-pq4hx_e55f3a24-f1c7-4556-a5eb-8a9ba6e89fe1/payload-processing/0.log" Sep 11 12:25:38.738244 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:38.738212 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5qrzc_d1190be0-add2-48ea-ad87-5d7271697bfa/serve-healthcheck-canary/0.log" Sep 11 12:25:39.416788 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:39.416758 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gncvx_55e0af20-17e2-45af-a9ae-06d4a206d839/kube-rbac-proxy/0.log" Sep 11 12:25:39.441792 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:39.441766 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gncvx_55e0af20-17e2-45af-a9ae-06d4a206d839/exporter/0.log" Sep 11 12:25:39.470382 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:39.466635 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gncvx_55e0af20-17e2-45af-a9ae-06d4a206d839/extractor/0.log" Sep 11 12:25:40.186394 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:40.186363 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-d2k5f" Sep 11 12:25:42.216408 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:42.216380 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-c4fd66f5c-lnlwl_df5f488c-1c66-462d-a1c3-b8abe43dd74f/manager/0.log" Sep 11 12:25:42.614545 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:42.614506 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fb8846894-zcf9q_77edd868-8a77-4e81-8bf9-301419e19c8d/manager/0.log" Sep 11 12:25:42.635643 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:42.635619 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fb8846894-zcf9q_77edd868-8a77-4e81-8bf9-301419e19c8d/copy-manifests/0.log" Sep 11 12:25:42.908188 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:42.908094 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5_63b1165a-60a3-4edc-b78d-af87251b58ec/istio-proxy/0.log" Sep 11 12:25:43.432877 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:43.432844 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-7v444_c888eade-3e00-4c36-b236-6c86b58d33bc/istio-proxy/0.log" Sep 11 12:25:43.439978 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:43.439956 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-7v444_c888eade-3e00-4c36-b236-6c86b58d33bc/istio-proxy/1.log" Sep 11 12:25:43.552196 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:43.552171 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-pq4hx_e55f3a24-f1c7-4556-a5eb-8a9ba6e89fe1/payload-processing/0.log" Sep 11 12:25:44.035812 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:44.035756 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-866dd874d6-zmnzz_b4943790-c1ba-4494-8f60-9b8a2160f5ab/manager/0.log" Sep 11 12:25:44.065639 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:44.065604 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-r4pk2_19402fa6-d6fd-456b-8e28-5a82dc39f678/openshift-lws-operator/0.log" Sep 11 12:25:45.117912 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:45.117884 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-rsnd5_63b1165a-60a3-4edc-b78d-af87251b58ec/istio-proxy/0.log" Sep 11 12:25:45.607232 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:45.607207 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-7v444_c888eade-3e00-4c36-b236-6c86b58d33bc/istio-proxy/0.log" Sep 11 12:25:45.614709 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:45.614675 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-7v444_c888eade-3e00-4c36-b236-6c86b58d33bc/istio-proxy/1.log" Sep 11 12:25:45.671072 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:45.671040 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-pq4hx_e55f3a24-f1c7-4556-a5eb-8a9ba6e89fe1/payload-processing/0.log" Sep 11 12:25:51.312599 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:51.312564 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cs8ww_091f9985-8446-4191-9ec0-9f40b1d1a5fb/kube-multus-additional-cni-plugins/0.log" Sep 11 12:25:51.336418 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:51.336366 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cs8ww_091f9985-8446-4191-9ec0-9f40b1d1a5fb/egress-router-binary-copy/0.log" Sep 11 12:25:51.359124 ip-10-0-132-170 kubenswrapper[2566]: I0911 12:25:51.359081 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cs8ww_091f9985-8446-4191-9ec0-9f40b1d1a5fb/cni-plugins/0.log"