Jul 28 17:06:33.390522 ip-10-0-132-11 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 28 17:06:33.390533 ip-10-0-132-11 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 28 17:06:33.390540 ip-10-0-132-11 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 28 17:06:33.390799 ip-10-0-132-11 systemd[1]: Failed to start Kubernetes Kubelet. Jul 28 17:06:43.627794 ip-10-0-132-11 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 28 17:06:43.627811 ip-10-0-132-11 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot eb712ec069114fe9a8b63d4a6d738fb4 -- Jul 28 17:08:21.264390 ip-10-0-132-11 systemd[1]: Starting Kubernetes Kubelet... Jul 28 17:08:21.703614 ip-10-0-132-11 kubenswrapper[2570]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 17:08:21.703614 ip-10-0-132-11 kubenswrapper[2570]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 28 17:08:21.703614 ip-10-0-132-11 kubenswrapper[2570]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 17:08:21.703614 ip-10-0-132-11 kubenswrapper[2570]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 28 17:08:21.703614 ip-10-0-132-11 kubenswrapper[2570]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 17:08:21.705778 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.705688 2570 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 28 17:08:21.710772 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710751 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:08:21.710772 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710768 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:08:21.710772 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710774 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:08:21.710974 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710779 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:08:21.710974 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710784 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:08:21.710974 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710788 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:08:21.710974 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710793 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:08:21.710974 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710796 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:08:21.710974 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710801 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:08:21.710974 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710804 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:08:21.710974 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710808 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:08:21.710974 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710812 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:08:21.710974 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710815 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:08:21.710974 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710819 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:08:21.710974 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710823 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:08:21.710974 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710826 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:08:21.710974 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710831 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:08:21.710974 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710834 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:08:21.710974 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710838 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:08:21.710974 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710842 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:08:21.710974 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710847 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:08:21.710974 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710850 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:08:21.710974 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710855 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:08:21.711759 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710859 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:08:21.711759 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710863 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:08:21.711759 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710867 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:08:21.711759 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710870 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:08:21.711759 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710874 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:08:21.711759 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710878 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:08:21.711759 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710883 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:08:21.711759 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710887 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:08:21.711759 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710891 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:08:21.711759 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710895 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:08:21.711759 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710900 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:08:21.711759 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710904 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:08:21.711759 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710908 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:08:21.711759 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710914 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:08:21.711759 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710918 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:08:21.711759 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710923 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:08:21.711759 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710928 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:08:21.711759 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710932 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:08:21.711759 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710937 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:08:21.711759 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710941 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:08:21.712302 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710945 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:08:21.712302 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710949 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:08:21.712302 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710954 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:08:21.712302 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710958 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:08:21.712302 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710974 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:08:21.712302 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710978 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:08:21.712302 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710982 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:08:21.712302 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710986 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:08:21.712302 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710990 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:08:21.712302 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710995 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:08:21.712302 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.710999 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:08:21.712302 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711006 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:08:21.712302 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711012 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:08:21.712302 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711017 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:08:21.712302 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711022 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:08:21.712302 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711028 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:08:21.712302 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711034 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:08:21.712302 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711040 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:08:21.712302 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711045 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:08:21.712917 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711050 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:08:21.712917 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711055 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:08:21.712917 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711059 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:08:21.712917 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711063 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:08:21.712917 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711070 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:08:21.712917 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711076 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:08:21.712917 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711082 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:08:21.712917 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711087 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:08:21.712917 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711093 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:08:21.712917 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711099 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:08:21.712917 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711103 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:08:21.712917 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711108 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:08:21.712917 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711127 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:08:21.712917 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711132 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:08:21.712917 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711136 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:08:21.712917 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711140 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:08:21.712917 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711144 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:08:21.712917 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711149 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:08:21.712917 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711153 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:08:21.712917 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711157 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:08:21.713782 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711162 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:08:21.713782 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711166 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:08:21.713782 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711170 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:08:21.713782 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711174 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:08:21.713782 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711178 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:08:21.713782 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711183 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:08:21.713782 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711187 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:08:21.713782 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711192 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:08:21.713782 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711196 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:08:21.713782 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711200 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:08:21.713782 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711205 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:08:21.713782 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711209 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:08:21.713782 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711213 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:08:21.713782 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711218 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:08:21.713782 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711223 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:08:21.713782 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711227 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:08:21.713782 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.711231 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:08:21.713782 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713353 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:08:21.713782 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713365 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:08:21.713782 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713370 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:08:21.713782 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713375 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:08:21.714691 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713380 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:08:21.714691 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713385 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:08:21.714691 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713390 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:08:21.714691 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713394 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:08:21.714691 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713399 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:08:21.714691 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713403 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:08:21.714691 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713407 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:08:21.714691 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713411 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:08:21.714691 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713415 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:08:21.714691 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713419 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:08:21.714691 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713424 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:08:21.714691 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713428 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:08:21.714691 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713433 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:08:21.714691 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713437 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:08:21.714691 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713441 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:08:21.714691 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713446 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:08:21.714691 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713450 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:08:21.714691 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713453 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:08:21.714691 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713458 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:08:21.714691 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713462 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:08:21.715589 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713467 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:08:21.715589 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713471 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:08:21.715589 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713476 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:08:21.715589 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713480 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:08:21.715589 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713485 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:08:21.715589 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713489 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:08:21.715589 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713493 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:08:21.715589 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713497 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:08:21.715589 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713501 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:08:21.715589 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713505 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:08:21.715589 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713512 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:08:21.715589 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713517 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:08:21.715589 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713522 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:08:21.715589 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713527 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:08:21.715589 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713532 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:08:21.715589 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713537 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:08:21.715589 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713541 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:08:21.715589 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713546 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:08:21.715589 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713550 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:08:21.715589 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713555 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:08:21.716483 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713559 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:08:21.716483 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713564 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:08:21.716483 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713568 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:08:21.716483 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713576 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:08:21.716483 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713582 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:08:21.716483 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713587 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:08:21.716483 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713593 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:08:21.716483 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713599 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:08:21.716483 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713603 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:08:21.716483 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713607 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:08:21.716483 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713611 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:08:21.716483 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713615 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:08:21.716483 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713620 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:08:21.716483 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713625 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:08:21.716483 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713630 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:08:21.716483 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713634 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:08:21.716483 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713638 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:08:21.716483 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713642 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:08:21.716483 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713647 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:08:21.716956 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713651 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:08:21.716956 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713655 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:08:21.716956 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713659 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:08:21.716956 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713663 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:08:21.716956 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713667 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:08:21.716956 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713672 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:08:21.716956 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713676 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:08:21.716956 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713680 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:08:21.716956 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713685 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:08:21.716956 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713689 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:08:21.716956 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713693 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:08:21.716956 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713697 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:08:21.716956 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713702 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:08:21.716956 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713707 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:08:21.716956 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713711 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:08:21.716956 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713716 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:08:21.716956 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713720 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:08:21.716956 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713725 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:08:21.716956 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713729 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:08:21.716956 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713733 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:08:21.717559 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713738 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:08:21.717559 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713741 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:08:21.717559 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713746 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:08:21.717559 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713750 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:08:21.717559 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713755 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:08:21.717559 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713762 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:08:21.717559 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713766 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:08:21.717559 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713770 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:08:21.717559 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713775 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:08:21.717559 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713778 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:08:21.717559 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713782 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:08:21.717559 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713786 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:08:21.717559 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713790 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:08:21.717559 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713794 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:08:21.717559 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713798 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:08:21.717559 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.713803 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:08:21.717559 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714420 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:08:21.717559 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714428 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:08:21.717559 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714435 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:08:21.717559 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714439 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:08:21.718052 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714444 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:08:21.718052 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714449 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:08:21.718052 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714453 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:08:21.718052 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714457 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:08:21.718052 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714461 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:08:21.718052 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714465 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:08:21.718052 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714469 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:08:21.718052 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714473 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:08:21.718052 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714478 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:08:21.718052 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714482 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:08:21.718052 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714487 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:08:21.718052 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714500 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:08:21.718052 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714505 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:08:21.718052 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714509 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:08:21.718052 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714513 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:08:21.718052 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714517 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:08:21.718052 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714522 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:08:21.718052 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714526 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:08:21.718052 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714531 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:08:21.718748 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714538 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:08:21.718748 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714544 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:08:21.718748 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714549 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:08:21.718748 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714553 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:08:21.718748 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714558 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:08:21.718748 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714563 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:08:21.718748 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714567 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:08:21.718748 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714573 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:08:21.718748 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714577 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:08:21.718748 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714581 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:08:21.718748 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714585 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:08:21.718748 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714589 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:08:21.718748 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714593 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:08:21.718748 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714597 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:08:21.718748 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714602 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:08:21.718748 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714606 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:08:21.718748 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714610 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:08:21.718748 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714614 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:08:21.718748 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714618 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:08:21.719277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714623 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:08:21.719277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714627 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:08:21.719277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714631 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:08:21.719277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714635 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:08:21.719277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714639 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:08:21.719277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714643 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:08:21.719277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714647 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:08:21.719277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714652 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:08:21.719277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714656 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:08:21.719277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714660 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:08:21.719277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714665 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:08:21.719277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714669 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:08:21.719277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714673 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:08:21.719277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714678 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:08:21.719277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714682 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:08:21.719277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714686 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:08:21.719277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714690 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:08:21.719277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714694 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:08:21.719277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714698 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:08:21.719277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714702 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:08:21.719864 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714706 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:08:21.719864 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714710 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:08:21.719864 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714714 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:08:21.719864 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714718 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:08:21.719864 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714722 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:08:21.719864 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714727 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:08:21.719864 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714731 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:08:21.719864 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714735 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:08:21.719864 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714740 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:08:21.719864 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714744 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:08:21.719864 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714748 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:08:21.719864 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714753 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:08:21.719864 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714758 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:08:21.719864 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714763 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:08:21.719864 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714767 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:08:21.719864 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714775 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:08:21.719864 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714779 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:08:21.719864 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714783 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:08:21.719864 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714787 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:08:21.719864 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714792 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:08:21.720400 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714796 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:08:21.720400 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714800 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:08:21.720400 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714804 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:08:21.720400 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714808 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:08:21.720400 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714812 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:08:21.720400 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714816 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:08:21.720400 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714821 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:08:21.720400 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714825 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:08:21.720400 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714828 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:08:21.720400 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714833 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:08:21.720400 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714837 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:08:21.720400 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714841 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:08:21.720400 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714845 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:08:21.720400 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714849 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:08:21.720400 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714853 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:08:21.720400 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714859 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:08:21.720400 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.714864 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:08:21.720400 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.714964 2570 flags.go:64] FLAG: --address="0.0.0.0" Jul 28 17:08:21.720400 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.714975 2570 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 28 17:08:21.720400 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.714984 2570 flags.go:64] FLAG: --anonymous-auth="true" Jul 28 17:08:21.720400 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.714991 2570 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 28 17:08:21.720964 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.714999 2570 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 28 17:08:21.720964 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715004 2570 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 28 17:08:21.720964 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715011 2570 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 28 17:08:21.720964 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715018 2570 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 28 17:08:21.720964 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715023 2570 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 28 17:08:21.720964 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715028 2570 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 28 17:08:21.720964 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715036 2570 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 28 17:08:21.720964 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715041 2570 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 28 17:08:21.720964 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715046 2570 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 28 17:08:21.720964 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715052 2570 flags.go:64] FLAG: --cgroup-root="" Jul 28 17:08:21.720964 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715056 2570 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 28 17:08:21.720964 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715062 2570 flags.go:64] FLAG: --client-ca-file="" Jul 28 17:08:21.720964 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715066 2570 flags.go:64] FLAG: --cloud-provider="external" Jul 28 17:08:21.720964 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715071 2570 flags.go:64] FLAG: --cluster-dns="[]" Jul 28 17:08:21.720964 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715078 2570 flags.go:64] FLAG: --cluster-domain="" Jul 28 17:08:21.720964 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715082 2570 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 28 17:08:21.720964 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715088 2570 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 28 17:08:21.720964 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715093 2570 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 28 17:08:21.720964 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715099 2570 flags.go:64] FLAG: --container-log-max-files="5" Jul 28 17:08:21.720964 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715104 2570 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 28 17:08:21.720964 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715125 2570 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 28 17:08:21.720964 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715130 2570 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 28 17:08:21.720964 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715135 2570 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 28 17:08:21.721539 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715140 2570 flags.go:64] FLAG: --contention-profiling="false" Jul 28 17:08:21.721539 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715145 2570 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 28 17:08:21.721539 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715150 2570 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 28 17:08:21.721539 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715155 2570 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 28 17:08:21.721539 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715160 2570 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 28 17:08:21.721539 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715166 2570 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 28 17:08:21.721539 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715171 2570 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 28 17:08:21.721539 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715175 2570 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 28 17:08:21.721539 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715180 2570 flags.go:64] FLAG: --enable-load-reader="false" Jul 28 17:08:21.721539 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715185 2570 flags.go:64] FLAG: --enable-server="true" Jul 28 17:08:21.721539 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715191 2570 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 28 17:08:21.721539 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715198 2570 flags.go:64] FLAG: --event-burst="100" Jul 28 17:08:21.721539 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715202 2570 flags.go:64] FLAG: --event-qps="50" Jul 28 17:08:21.721539 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715207 2570 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 28 17:08:21.721539 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715212 2570 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 28 17:08:21.721539 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715220 2570 flags.go:64] FLAG: --eviction-hard="" Jul 28 17:08:21.721539 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715226 2570 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 28 17:08:21.721539 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715231 2570 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 28 17:08:21.721539 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715237 2570 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 28 17:08:21.721539 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715242 2570 flags.go:64] FLAG: --eviction-soft="" Jul 28 17:08:21.721539 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715247 2570 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 28 17:08:21.721539 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715252 2570 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 28 17:08:21.721539 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715256 2570 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 28 17:08:21.721539 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715261 2570 flags.go:64] FLAG: --experimental-mounter-path="" Jul 28 17:08:21.721539 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715266 2570 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 28 17:08:21.722170 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715271 2570 flags.go:64] FLAG: --fail-swap-on="true" Jul 28 17:08:21.722170 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715276 2570 flags.go:64] FLAG: --feature-gates="" Jul 28 17:08:21.722170 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715282 2570 flags.go:64] FLAG: --file-check-frequency="20s" Jul 28 17:08:21.722170 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715287 2570 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 28 17:08:21.722170 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715292 2570 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 28 17:08:21.722170 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715297 2570 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 28 17:08:21.722170 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715302 2570 flags.go:64] FLAG: --healthz-port="10248" Jul 28 17:08:21.722170 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715307 2570 flags.go:64] FLAG: --help="false" Jul 28 17:08:21.722170 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715312 2570 flags.go:64] FLAG: --hostname-override="ip-10-0-132-11.ec2.internal" Jul 28 17:08:21.722170 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715317 2570 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 28 17:08:21.722170 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715322 2570 flags.go:64] FLAG: --http-check-frequency="20s" Jul 28 17:08:21.722170 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715327 2570 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 28 17:08:21.722170 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715332 2570 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 28 17:08:21.722170 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715338 2570 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 28 17:08:21.722170 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715343 2570 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 28 17:08:21.722170 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715347 2570 flags.go:64] FLAG: --image-service-endpoint="" Jul 28 17:08:21.722170 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715352 2570 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 28 17:08:21.722170 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715357 2570 flags.go:64] FLAG: --kube-api-burst="100" Jul 28 17:08:21.722170 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715363 2570 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 28 17:08:21.722170 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715369 2570 flags.go:64] FLAG: --kube-api-qps="50" Jul 28 17:08:21.722170 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715373 2570 flags.go:64] FLAG: --kube-reserved="" Jul 28 17:08:21.722170 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715378 2570 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 28 17:08:21.722170 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715385 2570 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 28 17:08:21.722170 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715390 2570 flags.go:64] FLAG: --kubelet-cgroups="" Jul 28 17:08:21.722776 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715395 2570 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 28 17:08:21.722776 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715401 2570 flags.go:64] FLAG: --lock-file="" Jul 28 17:08:21.722776 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715406 2570 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 28 17:08:21.722776 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715411 2570 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 28 17:08:21.722776 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715416 2570 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 28 17:08:21.722776 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715424 2570 flags.go:64] FLAG: --log-json-split-stream="false" Jul 28 17:08:21.722776 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715429 2570 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 28 17:08:21.722776 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715434 2570 flags.go:64] FLAG: --log-text-split-stream="false" Jul 28 17:08:21.722776 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715439 2570 flags.go:64] FLAG: --logging-format="text" Jul 28 17:08:21.722776 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715444 2570 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 28 17:08:21.722776 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715450 2570 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 28 17:08:21.722776 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715455 2570 flags.go:64] FLAG: --manifest-url="" Jul 28 17:08:21.722776 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715459 2570 flags.go:64] FLAG: --manifest-url-header="" Jul 28 17:08:21.722776 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715466 2570 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 28 17:08:21.722776 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715472 2570 flags.go:64] FLAG: --max-open-files="1000000" Jul 28 17:08:21.722776 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715478 2570 flags.go:64] FLAG: --max-pods="110" Jul 28 17:08:21.722776 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715483 2570 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 28 17:08:21.722776 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715488 2570 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 28 17:08:21.722776 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715493 2570 flags.go:64] FLAG: --memory-manager-policy="None" Jul 28 17:08:21.722776 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715497 2570 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 28 17:08:21.722776 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715502 2570 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 28 17:08:21.722776 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715507 2570 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 28 17:08:21.722776 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715511 2570 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 28 17:08:21.722776 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715523 2570 flags.go:64] FLAG: --node-status-max-images="50" Jul 28 17:08:21.722776 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715528 2570 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 28 17:08:21.723460 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715533 2570 flags.go:64] FLAG: --oom-score-adj="-999" Jul 28 17:08:21.723460 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715538 2570 flags.go:64] FLAG: --pod-cidr="" Jul 28 17:08:21.723460 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715543 2570 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 28 17:08:21.723460 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715552 2570 flags.go:64] FLAG: --pod-manifest-path="" Jul 28 17:08:21.723460 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715557 2570 flags.go:64] FLAG: --pod-max-pids="-1" Jul 28 17:08:21.723460 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715563 2570 flags.go:64] FLAG: --pods-per-core="0" Jul 28 17:08:21.723460 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715568 2570 flags.go:64] FLAG: --port="10250" Jul 28 17:08:21.723460 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715573 2570 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 28 17:08:21.723460 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715578 2570 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-018e4b13931ed2199" Jul 28 17:08:21.723460 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715585 2570 flags.go:64] FLAG: --qos-reserved="" Jul 28 17:08:21.723460 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715589 2570 flags.go:64] FLAG: --read-only-port="10255" Jul 28 17:08:21.723460 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715594 2570 flags.go:64] FLAG: --register-node="true" Jul 28 17:08:21.723460 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715599 2570 flags.go:64] FLAG: --register-with-taints="" Jul 28 17:08:21.723460 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715604 2570 flags.go:64] FLAG: --registry-burst="10" Jul 28 17:08:21.723460 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715608 2570 flags.go:64] FLAG: --registry-qps="5" Jul 28 17:08:21.723460 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715613 2570 flags.go:64] FLAG: --reserved-cpus="" Jul 28 17:08:21.723460 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715618 2570 flags.go:64] FLAG: --reserved-memory="" Jul 28 17:08:21.723460 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715624 2570 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 28 17:08:21.723460 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715630 2570 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 28 17:08:21.723460 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715635 2570 flags.go:64] FLAG: --rotate-certificates="false" Jul 28 17:08:21.723460 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715640 2570 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 28 17:08:21.723460 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715644 2570 flags.go:64] FLAG: --runonce="false" Jul 28 17:08:21.723460 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715649 2570 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 28 17:08:21.723460 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715654 2570 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 28 17:08:21.723460 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715659 2570 flags.go:64] FLAG: --seccomp-default="false" Jul 28 17:08:21.724081 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715663 2570 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 28 17:08:21.724081 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715668 2570 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 28 17:08:21.724081 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715673 2570 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 28 17:08:21.724081 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715678 2570 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 28 17:08:21.724081 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715684 2570 flags.go:64] FLAG: --storage-driver-password="root" Jul 28 17:08:21.724081 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715688 2570 flags.go:64] FLAG: --storage-driver-secure="false" Jul 28 17:08:21.724081 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715693 2570 flags.go:64] FLAG: --storage-driver-table="stats" Jul 28 17:08:21.724081 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715697 2570 flags.go:64] FLAG: --storage-driver-user="root" Jul 28 17:08:21.724081 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715702 2570 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 28 17:08:21.724081 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715707 2570 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 28 17:08:21.724081 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715713 2570 flags.go:64] FLAG: --system-cgroups="" Jul 28 17:08:21.724081 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715718 2570 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 28 17:08:21.724081 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715729 2570 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 28 17:08:21.724081 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715733 2570 flags.go:64] FLAG: --tls-cert-file="" Jul 28 17:08:21.724081 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715738 2570 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 28 17:08:21.724081 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715745 2570 flags.go:64] FLAG: --tls-min-version="" Jul 28 17:08:21.724081 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715751 2570 flags.go:64] FLAG: --tls-private-key-file="" Jul 28 17:08:21.724081 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715756 2570 flags.go:64] FLAG: --topology-manager-policy="none" Jul 28 17:08:21.724081 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715761 2570 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 28 17:08:21.724081 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715766 2570 flags.go:64] FLAG: --topology-manager-scope="container" Jul 28 17:08:21.724081 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715771 2570 flags.go:64] FLAG: --v="2" Jul 28 17:08:21.724081 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715777 2570 flags.go:64] FLAG: --version="false" Jul 28 17:08:21.724081 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715793 2570 flags.go:64] FLAG: --vmodule="" Jul 28 17:08:21.724081 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715800 2570 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 28 17:08:21.724081 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.715806 2570 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 28 17:08:21.724728 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.715983 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:08:21.724728 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.715990 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:08:21.724728 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.715995 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:08:21.724728 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716000 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:08:21.724728 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716005 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:08:21.724728 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716011 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:08:21.724728 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716016 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:08:21.724728 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716021 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:08:21.724728 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716025 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:08:21.724728 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716030 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:08:21.724728 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716034 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:08:21.724728 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716038 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:08:21.724728 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716043 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:08:21.724728 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716047 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:08:21.724728 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716052 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:08:21.724728 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716056 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:08:21.724728 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716062 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:08:21.724728 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716068 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:08:21.724728 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716074 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:08:21.724728 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716080 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:08:21.725277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716085 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:08:21.725277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716089 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:08:21.725277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716094 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:08:21.725277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716099 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:08:21.725277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716104 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:08:21.725277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716126 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:08:21.725277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716131 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:08:21.725277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716134 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:08:21.725277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716139 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:08:21.725277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716143 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:08:21.725277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716146 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:08:21.725277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716151 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:08:21.725277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716155 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:08:21.725277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716159 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:08:21.725277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716164 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:08:21.725277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716170 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:08:21.725277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716176 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:08:21.725277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716182 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:08:21.725277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716189 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:08:21.725738 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716194 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:08:21.725738 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716199 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:08:21.725738 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716204 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:08:21.725738 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716208 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:08:21.725738 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716212 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:08:21.725738 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716216 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:08:21.725738 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716221 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:08:21.725738 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716225 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:08:21.725738 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716229 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:08:21.725738 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716233 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:08:21.725738 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716237 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:08:21.725738 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716242 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:08:21.725738 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716248 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:08:21.725738 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716253 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:08:21.725738 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716257 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:08:21.725738 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716261 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:08:21.725738 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716267 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:08:21.725738 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716271 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:08:21.725738 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716275 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:08:21.725738 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716279 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:08:21.726237 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716283 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:08:21.726237 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716287 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:08:21.726237 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716291 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:08:21.726237 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716296 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:08:21.726237 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716300 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:08:21.726237 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716304 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:08:21.726237 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716309 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:08:21.726237 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716313 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:08:21.726237 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716317 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:08:21.726237 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716321 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:08:21.726237 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716326 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:08:21.726237 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716330 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:08:21.726237 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716334 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:08:21.726237 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716338 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:08:21.726237 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716342 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:08:21.726237 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716346 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:08:21.726237 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716351 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:08:21.726237 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716355 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:08:21.726237 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716359 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:08:21.726237 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716363 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:08:21.726997 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716367 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:08:21.726997 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716371 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:08:21.726997 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716376 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:08:21.726997 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716380 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:08:21.726997 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716385 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:08:21.726997 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716394 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:08:21.726997 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716399 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:08:21.726997 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716403 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:08:21.726997 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716409 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:08:21.726997 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716413 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:08:21.726997 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716418 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:08:21.726997 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716423 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:08:21.726997 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716427 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:08:21.726997 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716431 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:08:21.726997 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716435 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:08:21.726997 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716439 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:08:21.726997 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716443 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:08:21.726997 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716447 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:08:21.726997 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716451 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:08:21.726997 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.716456 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:08:21.727694 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.717248 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 17:08:21.727694 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.724570 2570 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 28 17:08:21.727694 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.724585 2570 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 28 17:08:21.727694 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.725557 2570 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 28 17:08:21.727694 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.725566 2570 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 28 17:08:21.727694 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725620 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:08:21.727694 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725625 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:08:21.727694 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725629 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:08:21.727694 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725631 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:08:21.727694 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725635 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:08:21.727694 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725638 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:08:21.727694 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725641 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:08:21.727694 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725644 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:08:21.727694 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725646 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:08:21.727694 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725649 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:08:21.728066 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725651 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:08:21.728066 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725654 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:08:21.728066 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725657 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:08:21.728066 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725660 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:08:21.728066 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725662 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:08:21.728066 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725665 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:08:21.728066 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725667 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:08:21.728066 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725670 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:08:21.728066 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725672 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:08:21.728066 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725675 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:08:21.728066 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725678 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:08:21.728066 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725680 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:08:21.728066 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725684 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:08:21.728066 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725688 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:08:21.728066 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725691 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:08:21.728066 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725694 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:08:21.728066 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725697 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:08:21.728066 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725699 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:08:21.728066 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725702 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:08:21.728550 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725707 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:08:21.728550 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725710 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:08:21.728550 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725713 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:08:21.728550 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725715 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:08:21.728550 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725718 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:08:21.728550 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725721 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:08:21.728550 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725724 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:08:21.728550 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725727 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:08:21.728550 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725729 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:08:21.728550 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725732 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:08:21.728550 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725734 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:08:21.728550 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725753 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:08:21.728550 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725758 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:08:21.728550 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725761 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:08:21.728550 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725764 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:08:21.728550 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725767 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:08:21.728550 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725770 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:08:21.728550 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725773 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:08:21.728550 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725776 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:08:21.728550 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725778 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:08:21.729024 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725781 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:08:21.729024 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725783 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:08:21.729024 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725786 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:08:21.729024 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725788 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:08:21.729024 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725792 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:08:21.729024 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725794 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:08:21.729024 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725797 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:08:21.729024 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725800 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:08:21.729024 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725802 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:08:21.729024 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725805 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:08:21.729024 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725808 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:08:21.729024 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725810 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:08:21.729024 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725813 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:08:21.729024 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725816 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:08:21.729024 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725819 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:08:21.729024 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725821 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:08:21.729024 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725824 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:08:21.729024 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725826 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:08:21.729024 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725829 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:08:21.729024 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725831 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:08:21.729591 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725834 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:08:21.729591 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725836 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:08:21.729591 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725838 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:08:21.729591 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725841 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:08:21.729591 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725843 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:08:21.729591 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725846 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:08:21.729591 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725848 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:08:21.729591 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725851 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:08:21.729591 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725853 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:08:21.729591 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725856 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:08:21.729591 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725858 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:08:21.729591 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725861 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:08:21.729591 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725863 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:08:21.729591 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725866 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:08:21.729591 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725868 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:08:21.729591 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725871 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:08:21.729591 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725873 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:08:21.729591 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725876 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:08:21.729591 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725878 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:08:21.729591 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725881 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:08:21.730070 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725884 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:08:21.730070 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725887 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:08:21.730070 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725889 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:08:21.730070 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725891 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:08:21.730070 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725894 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:08:21.730070 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725896 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:08:21.730070 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725900 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:08:21.730070 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725904 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:08:21.730070 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725906 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:08:21.730070 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.725909 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:08:21.730070 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.725914 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 17:08:21.730070 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726025 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:08:21.730070 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726029 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:08:21.730070 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726032 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:08:21.730070 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726035 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:08:21.730459 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726037 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:08:21.730459 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726040 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:08:21.730459 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726042 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:08:21.730459 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726045 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:08:21.730459 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726047 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:08:21.730459 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726049 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:08:21.730459 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726052 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:08:21.730459 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726054 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:08:21.730459 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726057 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:08:21.730459 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726059 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:08:21.730459 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726062 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:08:21.730459 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726064 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:08:21.730459 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726067 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:08:21.730459 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726070 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:08:21.730459 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726072 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:08:21.730459 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726075 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:08:21.730459 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726077 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:08:21.730459 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726080 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:08:21.730459 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726083 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:08:21.730459 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726086 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:08:21.730936 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726090 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:08:21.730936 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726093 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:08:21.730936 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726096 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:08:21.730936 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726099 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:08:21.730936 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726103 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:08:21.730936 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726105 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:08:21.730936 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726122 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:08:21.730936 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726126 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:08:21.730936 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726129 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:08:21.730936 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726132 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:08:21.730936 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726135 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:08:21.730936 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726137 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:08:21.730936 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726140 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:08:21.730936 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726143 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:08:21.730936 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726145 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:08:21.730936 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726148 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:08:21.730936 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726151 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:08:21.730936 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726153 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:08:21.730936 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726156 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:08:21.730936 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726158 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:08:21.731432 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726161 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:08:21.731432 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726164 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:08:21.731432 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726167 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:08:21.731432 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726171 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:08:21.731432 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726174 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:08:21.731432 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726176 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:08:21.731432 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726179 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:08:21.731432 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726182 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:08:21.731432 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726185 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:08:21.731432 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726187 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:08:21.731432 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726190 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:08:21.731432 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726192 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:08:21.731432 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726195 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:08:21.731432 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726197 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:08:21.731432 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726200 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:08:21.731432 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726202 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:08:21.731432 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726205 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:08:21.731432 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726208 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:08:21.731432 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726211 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:08:21.731885 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726213 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:08:21.731885 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726216 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:08:21.731885 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726218 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:08:21.731885 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726221 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:08:21.731885 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726223 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:08:21.731885 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726225 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:08:21.731885 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726228 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:08:21.731885 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726230 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:08:21.731885 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726233 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:08:21.731885 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726235 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:08:21.731885 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726238 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:08:21.731885 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726240 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:08:21.731885 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726243 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:08:21.731885 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726246 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:08:21.731885 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726248 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:08:21.731885 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726251 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:08:21.731885 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726254 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:08:21.731885 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726256 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:08:21.731885 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726258 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:08:21.731885 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726261 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:08:21.732392 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726263 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:08:21.732392 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726266 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:08:21.732392 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726269 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:08:21.732392 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726271 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:08:21.732392 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726274 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:08:21.732392 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726276 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:08:21.732392 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726279 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:08:21.732392 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726281 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:08:21.732392 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726283 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:08:21.732392 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726286 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:08:21.732392 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726289 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:08:21.732392 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726326 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:08:21.732392 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726376 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:08:21.732392 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726382 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:08:21.732392 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726385 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:08:21.732392 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:21.726389 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:08:21.732771 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.726395 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 17:08:21.732771 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.727767 2570 server.go:962] "Client rotation is on, will bootstrap in background" Jul 28 17:08:21.732771 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.730197 2570 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 28 17:08:21.732771 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.731934 2570 server.go:1019] "Starting client certificate rotation" Jul 28 17:08:21.732771 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.732041 2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 28 17:08:21.732915 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.732847 2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 28 17:08:21.734658 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.734644 2570 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 28 17:08:21.737730 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.737715 2570 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 28 17:08:21.748141 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.748124 2570 log.go:25] "Validated CRI v1 runtime API" Jul 28 17:08:21.753274 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.753258 2570 log.go:25] "Validated CRI v1 image API" Jul 28 17:08:21.754107 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.754092 2570 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 28 17:08:21.757348 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.757321 2570 fs.go:135] Filesystem UUIDs: map[420b2d3a-a2d3-4fa0-a524-afd5ca464c94:/dev/nvme0n1p3 4a6d866b-609f-4070-acfe-6d02ddfff766:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 28 17:08:21.757417 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.757342 2570 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 28 17:08:21.763089 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.762982 2570 manager.go:216] Machine: {Timestamp:2026-07-28 17:08:21.761518473 +0000 UTC m=+0.370449834 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3088338 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c86fbf75d03b7b9a015dd69a1d20a SystemUUID:ec2c86fb-f75d-03b7-b9a0-15dd69a1d20a BootID:eb712ec0-6911-4fe9-a8b6-3d4a6d738fb4 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:40:89:f2:20:d3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:40:89:f2:20:d3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0e:3c:32:af:9b:f4 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 28 17:08:21.763089 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.763083 2570 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 28 17:08:21.763231 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.763171 2570 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 28 17:08:21.766633 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.766601 2570 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 28 17:08:21.766793 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.766639 2570 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-11.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 28 17:08:21.767406 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.767391 2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 28 17:08:21.767539 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.767523 2570 topology_manager.go:138] "Creating topology manager with none policy" Jul 28 17:08:21.767581 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.767543 2570 container_manager_linux.go:309] "Creating device plugin manager" Jul 28 17:08:21.767581 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.767550 2570 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 28 17:08:21.767581 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.767566 2570 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 28 17:08:21.768874 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.768864 2570 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 28 17:08:21.771772 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.771760 2570 state_mem.go:36] "Initialized new in-memory state store" Jul 28 17:08:21.771898 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.771888 2570 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 28 17:08:21.771939 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.771916 2570 kubelet.go:486] "Attempting to sync node with API server" Jul 28 17:08:21.771966 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.771939 2570 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 28 17:08:21.771966 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.771963 2570 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 28 17:08:21.772038 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.772027 2570 kubelet.go:398] "Adding apiserver pod source" Jul 28 17:08:21.772038 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.772036 2570 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 28 17:08:21.772607 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.772591 2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 28 17:08:21.772681 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.772611 2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 28 17:08:21.775263 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.775242 2570 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 28 17:08:21.778792 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.778776 2570 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 28 17:08:21.778868 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.778807 2570 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 28 17:08:21.780784 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.780772 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 28 17:08:21.780835 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.780789 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 28 17:08:21.780835 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.780795 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 28 17:08:21.780835 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.780800 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 28 17:08:21.780835 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.780808 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 28 17:08:21.780835 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.780813 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 28 17:08:21.780835 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.780824 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 28 17:08:21.780835 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.780831 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 28 17:08:21.781017 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.780839 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 28 17:08:21.781017 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.780844 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 28 17:08:21.781017 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.780853 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 28 17:08:21.781017 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.780858 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 28 17:08:21.781696 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.781685 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 28 17:08:21.781726 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.781700 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 28 17:08:21.784859 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.784841 2570 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-11.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 28 17:08:21.784937 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.784886 2570 server.go:1268] "Started kubelet" Jul 28 17:08:21.784937 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:21.784889 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 28 17:08:21.785089 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.785030 2570 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 28 17:08:21.785169 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.785127 2570 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 28 17:08:21.785169 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:21.784884 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-11.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 28 17:08:21.785712 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.785663 2570 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 28 17:08:21.785710 ip-10-0-132-11 systemd[1]: Started Kubernetes Kubelet. Jul 28 17:08:21.787655 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.787642 2570 server.go:310] "Adding debug handlers to kubelet server" Jul 28 17:08:21.790628 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.790610 2570 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 28 17:08:21.791021 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:21.791000 2570 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 28 17:08:21.795789 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:21.794568 2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-11.ec2.internal.18c683260f21faec default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-11.ec2.internal,UID:ip-10-0-132-11.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-11.ec2.internal,},FirstTimestamp:2026-07-28 17:08:21.78486142 +0000 UTC m=+0.393792787,LastTimestamp:2026-07-28 17:08:21.78486142 +0000 UTC m=+0.393792787,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-11.ec2.internal,}" Jul 28 17:08:21.799394 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.799376 2570 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 28 17:08:21.799488 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.799392 2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 28 17:08:21.799488 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:21.799465 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-11.ec2.internal\" not found" Jul 28 17:08:21.800239 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.800207 2570 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 28 17:08:21.800332 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.800286 2570 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 28 17:08:21.800332 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.800309 2570 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 28 17:08:21.800416 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.800402 2570 factory.go:55] Registering systemd factory Jul 28 17:08:21.800448 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.800423 2570 factory.go:223] Registration of the systemd container factory successfully Jul 28 17:08:21.800448 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.800437 2570 reconstruct.go:97] "Volume reconstruction finished" Jul 28 17:08:21.800448 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.800447 2570 reconciler.go:29] "Reconciler: start to sync state" Jul 28 17:08:21.800665 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.800644 2570 factory.go:153] Registering CRI-O factory Jul 28 17:08:21.800665 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.800661 2570 factory.go:223] Registration of the crio container factory successfully Jul 28 17:08:21.800782 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.800717 2570 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 28 17:08:21.800782 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.800743 2570 factory.go:103] Registering Raw factory Jul 28 17:08:21.800782 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.800763 2570 manager.go:1196] Started watching for new ooms in manager Jul 28 17:08:21.801172 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.801160 2570 manager.go:349] Starting recovery of all containers Jul 28 17:08:21.803558 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:21.803538 2570 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-11.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 28 17:08:21.803661 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:21.803641 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 28 17:08:21.816106 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.816086 2570 manager.go:354] Recovery completed Jul 28 17:08:21.820188 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.820175 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:08:21.821951 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.821924 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:08:21.822007 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.821969 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:08:21.822007 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.821984 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:08:21.822592 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.822578 2570 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 28 17:08:21.822592 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.822593 2570 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 28 17:08:21.822701 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.822611 2570 state_mem.go:36] "Initialized new in-memory state store" Jul 28 17:08:21.822755 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.822748 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:08:21.823627 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.823607 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:08:21.823701 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.823639 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:08:21.823701 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.823658 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:08:21.824520 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:21.824442 2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-11.ec2.internal.18c683261157f0c7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-11.ec2.internal,UID:ip-10-0-132-11.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-11.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-11.ec2.internal,},FirstTimestamp:2026-07-28 17:08:21.821952199 +0000 UTC m=+0.430883564,LastTimestamp:2026-07-28 17:08:21.821952199 +0000 UTC m=+0.430883564,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-11.ec2.internal,}" Jul 28 17:08:21.825475 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.825457 2570 policy_none.go:49] "None policy: Start" Jul 28 17:08:21.825475 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.825478 2570 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 28 17:08:21.825583 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.825489 2570 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 28 17:08:21.827357 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.827346 2570 policy_none.go:47] "Start" Jul 28 17:08:21.837077 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:21.836990 2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-11.ec2.internal.18c6832611584fa1 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-11.ec2.internal,UID:ip-10-0-132-11.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-132-11.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-132-11.ec2.internal,},FirstTimestamp:2026-07-28 17:08:21.821976481 +0000 UTC m=+0.430907843,LastTimestamp:2026-07-28 17:08:21.821976481 +0000 UTC m=+0.430907843,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-11.ec2.internal,}" Jul 28 17:08:21.847148 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:21.845945 2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-11.ec2.internal.18c6832611588482 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-11.ec2.internal,UID:ip-10-0-132-11.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-132-11.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-132-11.ec2.internal,},FirstTimestamp:2026-07-28 17:08:21.821990018 +0000 UTC m=+0.430921380,LastTimestamp:2026-07-28 17:08:21.821990018 +0000 UTC m=+0.430921380,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-11.ec2.internal,}" Jul 28 17:08:21.853225 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.853204 2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qst7q" Jul 28 17:08:21.859288 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:21.859226 2570 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-132-11.ec2.internal.18c683261157f0c7\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-11.ec2.internal.18c683261157f0c7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-11.ec2.internal,UID:ip-10-0-132-11.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-11.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-11.ec2.internal,},FirstTimestamp:2026-07-28 17:08:21.821952199 +0000 UTC m=+0.430883564,LastTimestamp:2026-07-28 17:08:21.823624454 +0000 UTC m=+0.432555816,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-11.ec2.internal,}" Jul 28 17:08:21.862951 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.862935 2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qst7q" Jul 28 17:08:21.873317 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.873305 2570 manager.go:337] "Starting Device Plugin manager" Jul 28 17:08:21.873389 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:21.873339 2570 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 28 17:08:21.873389 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.873350 2570 server.go:87] "Starting device plugin registration server" Jul 28 17:08:21.873608 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.873595 2570 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 28 17:08:21.873651 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.873613 2570 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 28 17:08:21.873767 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.873751 2570 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 28 17:08:21.873838 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.873826 2570 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 28 17:08:21.873885 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.873840 2570 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 28 17:08:21.874281 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:21.874251 2570 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 28 17:08:21.874356 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:21.874301 2570 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-11.ec2.internal\" not found" Jul 28 17:08:21.902408 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.902390 2570 container_manager_linux.go:676] "Failed to ensure state" containerName="/system.slice" err="failed to move PID 2602 into the system container \"/system.slice\": " Jul 28 17:08:21.974272 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.974204 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:08:21.975075 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.975057 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:08:21.975186 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.975094 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:08:21.975186 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.975105 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:08:21.975186 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.975148 2570 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-11.ec2.internal" Jul 28 17:08:21.985243 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:21.985214 2570 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-11.ec2.internal" Jul 28 17:08:21.985243 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:21.985237 2570 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-11.ec2.internal\": node \"ip-10-0-132-11.ec2.internal\" not found" Jul 28 17:08:22.018015 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:22.017967 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-11.ec2.internal\" not found" Jul 28 17:08:22.111703 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.111675 2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 28 17:08:22.113008 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.112992 2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 28 17:08:22.113082 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.113029 2570 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 28 17:08:22.113082 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.113049 2570 kubelet.go:2448] "Starting kubelet main sync loop" Jul 28 17:08:22.113185 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:22.113083 2570 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 28 17:08:22.115831 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.115811 2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 17:08:22.119029 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:22.119011 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-11.ec2.internal\" not found" Jul 28 17:08:22.213731 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.213557 2570 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-132-11.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal"] Jul 28 17:08:22.213859 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.213770 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:08:22.214760 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.214744 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:08:22.214849 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.214780 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:08:22.214849 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.214796 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:08:22.217077 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.217061 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:08:22.217186 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.217172 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-11.ec2.internal" Jul 28 17:08:22.217251 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.217199 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:08:22.217786 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.217770 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:08:22.217882 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.217801 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:08:22.217882 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.217817 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:08:22.217882 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.217855 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:08:22.217882 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.217872 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:08:22.217882 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.217881 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:08:22.219161 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:22.219143 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-11.ec2.internal\" not found" Jul 28 17:08:22.220075 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.220060 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal" Jul 28 17:08:22.220186 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.220086 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:08:22.220864 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.220850 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:08:22.220934 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.220876 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:08:22.220934 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.220886 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:08:22.250718 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:22.250672 2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-11.ec2.internal\" not found" node="ip-10-0-132-11.ec2.internal" Jul 28 17:08:22.255088 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:22.255074 2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-11.ec2.internal\" not found" node="ip-10-0-132-11.ec2.internal" Jul 28 17:08:22.303725 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.303677 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29924a4ae605b9b2aa658a2296cc479f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal\" (UID: \"29924a4ae605b9b2aa658a2296cc479f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal" Jul 28 17:08:22.303834 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.303730 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a8cc528b9c87d3a6a58c64430cacff3d-config\") pod \"kube-apiserver-proxy-ip-10-0-132-11.ec2.internal\" (UID: \"a8cc528b9c87d3a6a58c64430cacff3d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-11.ec2.internal" Jul 28 17:08:22.303834 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.303753 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/29924a4ae605b9b2aa658a2296cc479f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal\" (UID: \"29924a4ae605b9b2aa658a2296cc479f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal" Jul 28 17:08:22.319325 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:22.319308 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-11.ec2.internal\" not found" Jul 28 17:08:22.404836 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.404798 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/29924a4ae605b9b2aa658a2296cc479f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal\" (UID: \"29924a4ae605b9b2aa658a2296cc479f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal" Jul 28 17:08:22.404836 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.404845 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29924a4ae605b9b2aa658a2296cc479f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal\" (UID: \"29924a4ae605b9b2aa658a2296cc479f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal" Jul 28 17:08:22.404991 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.404862 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a8cc528b9c87d3a6a58c64430cacff3d-config\") pod \"kube-apiserver-proxy-ip-10-0-132-11.ec2.internal\" (UID: \"a8cc528b9c87d3a6a58c64430cacff3d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-11.ec2.internal" Jul 28 17:08:22.404991 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.404916 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29924a4ae605b9b2aa658a2296cc479f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal\" (UID: \"29924a4ae605b9b2aa658a2296cc479f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal" Jul 28 17:08:22.404991 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.404923 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/29924a4ae605b9b2aa658a2296cc479f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal\" (UID: \"29924a4ae605b9b2aa658a2296cc479f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal" Jul 28 17:08:22.404991 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.404975 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a8cc528b9c87d3a6a58c64430cacff3d-config\") pod \"kube-apiserver-proxy-ip-10-0-132-11.ec2.internal\" (UID: \"a8cc528b9c87d3a6a58c64430cacff3d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-11.ec2.internal" Jul 28 17:08:22.419916 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:22.419898 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-11.ec2.internal\" not found" Jul 28 17:08:22.520518 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:22.520457 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-11.ec2.internal\" not found" Jul 28 17:08:22.553694 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.553668 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-11.ec2.internal" Jul 28 17:08:22.558645 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.558440 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal" Jul 28 17:08:22.620763 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:22.620728 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-11.ec2.internal\" not found" Jul 28 17:08:22.721334 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:22.721279 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-11.ec2.internal\" not found" Jul 28 17:08:22.731166 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.731140 2570 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 28 17:08:22.731308 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.731291 2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 28 17:08:22.821823 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:22.821751 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-11.ec2.internal\" not found" Jul 28 17:08:22.871940 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.871897 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 17:03:22 +0000 UTC" deadline="2028-02-10 16:19:35.800083395 +0000 UTC" Jul 28 17:08:22.871940 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.871931 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13487h11m12.928155611s" Jul 28 17:08:22.887043 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.887020 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:08:22.888084 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.888068 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:08:22.888187 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.888101 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:08:22.888187 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.888112 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:08:22.922175 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:22.922133 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-11.ec2.internal\" not found" Jul 28 17:08:22.966952 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:22.966926 2570 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 17:08:23.000408 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.000373 2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal" Jul 28 17:08:23.033967 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.033942 2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 28 17:08:23.035493 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.035479 2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-11.ec2.internal" Jul 28 17:08:23.055932 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.055908 2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 28 17:08:23.079402 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.079340 2570 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 17:08:23.216254 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:23.216219 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8cc528b9c87d3a6a58c64430cacff3d.slice/crio-c8cf06ab9de0018da8a22ca2448cb3c3a082bf9fd4bcb352f36620ff369f4e9e WatchSource:0}: Error finding container c8cf06ab9de0018da8a22ca2448cb3c3a082bf9fd4bcb352f36620ff369f4e9e: Status 404 returned error can't find the container with id c8cf06ab9de0018da8a22ca2448cb3c3a082bf9fd4bcb352f36620ff369f4e9e Jul 28 17:08:23.226093 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.226068 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 17:08:23.233334 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:23.233311 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29924a4ae605b9b2aa658a2296cc479f.slice/crio-f34963d1310f4af3e41eb5a10a0028f345b134c05e0e31e74c2bc2c7616ed23b WatchSource:0}: Error finding container f34963d1310f4af3e41eb5a10a0028f345b134c05e0e31e74c2bc2c7616ed23b: Status 404 returned error can't find the container with id f34963d1310f4af3e41eb5a10a0028f345b134c05e0e31e74c2bc2c7616ed23b Jul 28 17:08:23.355632 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.355541 2570 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 17:08:23.676772 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.676740 2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 17:08:23.774888 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.774859 2570 apiserver.go:52] "Watching apiserver" Jul 28 17:08:23.783141 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.783111 2570 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 28 17:08:23.783632 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.783603 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-cmx6g","openshift-multus/multus-additional-cni-plugins-b9mxn","openshift-multus/multus-jm8ng","openshift-network-diagnostics/network-check-target-6dbsb","openshift-network-operator/iptables-alerter-vmdd2","openshift-ovn-kubernetes/ovnkube-node-x6gbq","kube-system/global-pull-secret-syncer-fbzsz","kube-system/konnectivity-agent-dmc56","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksszp","openshift-cluster-node-tuning-operator/tuned-pf6xn","openshift-dns/node-resolver-zbmdl","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal","openshift-multus/network-metrics-daemon-5tdlw","kube-system/kube-apiserver-proxy-ip-10-0-132-11.ec2.internal"] Jul 28 17:08:23.786519 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.786498 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksszp" Jul 28 17:08:23.788988 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.788966 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-b9mxn" Jul 28 17:08:23.789869 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.789848 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 28 17:08:23.789994 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.789866 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-dfqt4\"" Jul 28 17:08:23.790329 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.790303 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 28 17:08:23.790439 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.790427 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 28 17:08:23.791455 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.791435 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.791793 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.791774 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 28 17:08:23.793907 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.793886 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6dbsb" Jul 28 17:08:23.794008 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:23.793982 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6dbsb" podUID="980a4184-ed46-42a1-99f3-86c49cfba43a" Jul 28 17:08:23.794487 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.794464 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 28 17:08:23.794637 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.794519 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 28 17:08:23.794637 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.794583 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 28 17:08:23.794778 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.794678 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 28 17:08:23.794847 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.794829 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-fcnc9\"" Jul 28 17:08:23.794931 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.794917 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-87jcl\"" Jul 28 17:08:23.796162 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.796141 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 28 17:08:23.797445 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.797355 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tdlw" Jul 28 17:08:23.797445 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:23.797428 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5tdlw" podUID="93b6b3b5-e860-4fee-9800-0f4534b72e67" Jul 28 17:08:23.799732 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.799703 2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 28 17:08:23.799838 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.799823 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zbmdl" Jul 28 17:08:23.802194 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.802150 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cmx6g" Jul 28 17:08:23.802711 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.802688 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 28 17:08:23.803545 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.803520 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-n6fwb\"" Jul 28 17:08:23.803644 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.803607 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 28 17:08:23.804751 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.804730 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dmc56" Jul 28 17:08:23.805059 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.804876 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 28 17:08:23.805217 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.805195 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 28 17:08:23.805417 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.805393 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-xnsvg\"" Jul 28 17:08:23.805965 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.805945 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 28 17:08:23.808076 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.808058 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbzsz" Jul 28 17:08:23.808201 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:23.808106 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fbzsz" podUID="9baa058e-a765-4c65-b5a6-ce0a7c022d97" Jul 28 17:08:23.808990 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.808972 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 28 17:08:23.809227 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.809212 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 28 17:08:23.809805 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.809773 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-wkj9g\"" Jul 28 17:08:23.810658 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.810611 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:23.811442 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.811367 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-multus-cni-dir\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.811442 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.811413 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/36c66dd3-6c06-4883-8ccd-107c566a45d6-system-cni-dir\") pod \"multus-additional-cni-plugins-b9mxn\" (UID: \"36c66dd3-6c06-4883-8ccd-107c566a45d6\") " pod="openshift-multus/multus-additional-cni-plugins-b9mxn" Jul 28 17:08:23.811595 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.811448 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/36c66dd3-6c06-4883-8ccd-107c566a45d6-cnibin\") pod \"multus-additional-cni-plugins-b9mxn\" (UID: \"36c66dd3-6c06-4883-8ccd-107c566a45d6\") " pod="openshift-multus/multus-additional-cni-plugins-b9mxn" Jul 28 17:08:23.811595 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.811474 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/56d1a325-976b-4060-b1a4-1a4b6be0bd18-host\") pod \"node-ca-cmx6g\" (UID: \"56d1a325-976b-4060-b1a4-1a4b6be0bd18\") " pod="openshift-image-registry/node-ca-cmx6g" Jul 28 17:08:23.811595 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.811507 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/56d1a325-976b-4060-b1a4-1a4b6be0bd18-serviceca\") pod \"node-ca-cmx6g\" (UID: \"56d1a325-976b-4060-b1a4-1a4b6be0bd18\") " pod="openshift-image-registry/node-ca-cmx6g" Jul 28 17:08:23.811595 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.811539 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/36c66dd3-6c06-4883-8ccd-107c566a45d6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b9mxn\" (UID: \"36c66dd3-6c06-4883-8ccd-107c566a45d6\") " pod="openshift-multus/multus-additional-cni-plugins-b9mxn" Jul 28 17:08:23.811799 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.811569 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-os-release\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.811799 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.811633 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lzzz\" (UniqueName: \"kubernetes.io/projected/980a4184-ed46-42a1-99f3-86c49cfba43a-kube-api-access-7lzzz\") pod \"network-check-target-6dbsb\" (UID: \"980a4184-ed46-42a1-99f3-86c49cfba43a\") " pod="openshift-network-diagnostics/network-check-target-6dbsb" Jul 28 17:08:23.812297 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.812264 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/36c66dd3-6c06-4883-8ccd-107c566a45d6-os-release\") pod \"multus-additional-cni-plugins-b9mxn\" (UID: \"36c66dd3-6c06-4883-8ccd-107c566a45d6\") " pod="openshift-multus/multus-additional-cni-plugins-b9mxn" Jul 28 17:08:23.812388 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.812332 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-host-var-lib-cni-bin\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.812388 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.812382 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-host-var-lib-cni-multus\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.812527 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.812408 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgvp5\" (UniqueName: \"kubernetes.io/projected/dd54c89e-8c35-404b-8804-ca3e45d384b4-kube-api-access-xgvp5\") pod \"node-resolver-zbmdl\" (UID: \"dd54c89e-8c35-404b-8804-ca3e45d384b4\") " pod="openshift-dns/node-resolver-zbmdl" Jul 28 17:08:23.812527 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.812444 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/edd0a7b9-158c-4fdf-8069-5e49e0ce0772-sys-fs\") pod \"aws-ebs-csi-driver-node-ksszp\" (UID: \"edd0a7b9-158c-4fdf-8069-5e49e0ce0772\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksszp" Jul 28 17:08:23.812527 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.812490 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-multus-conf-dir\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.812687 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.812530 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/75339f13-4483-4c0f-a502-885f0b22c8b7-multus-daemon-config\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.812687 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.812558 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-etc-kubernetes\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.812687 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.812597 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/36c66dd3-6c06-4883-8ccd-107c566a45d6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b9mxn\" (UID: \"36c66dd3-6c06-4883-8ccd-107c566a45d6\") " pod="openshift-multus/multus-additional-cni-plugins-b9mxn" Jul 28 17:08:23.812687 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.812636 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/75339f13-4483-4c0f-a502-885f0b22c8b7-cni-binary-copy\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.812866 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.812712 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-host-run-netns\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.812866 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.812743 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-hostroot\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.812866 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.812778 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dd54c89e-8c35-404b-8804-ca3e45d384b4-hosts-file\") pod \"node-resolver-zbmdl\" (UID: \"dd54c89e-8c35-404b-8804-ca3e45d384b4\") " pod="openshift-dns/node-resolver-zbmdl" Jul 28 17:08:23.812866 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.812804 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/edd0a7b9-158c-4fdf-8069-5e49e0ce0772-etc-selinux\") pod \"aws-ebs-csi-driver-node-ksszp\" (UID: \"edd0a7b9-158c-4fdf-8069-5e49e0ce0772\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksszp" Jul 28 17:08:23.813007 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.812855 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nn58s\" (UniqueName: \"kubernetes.io/projected/edd0a7b9-158c-4fdf-8069-5e49e0ce0772-kube-api-access-nn58s\") pod \"aws-ebs-csi-driver-node-ksszp\" (UID: \"edd0a7b9-158c-4fdf-8069-5e49e0ce0772\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksszp" Jul 28 17:08:23.813789 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.813296 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/36c66dd3-6c06-4883-8ccd-107c566a45d6-cni-binary-copy\") pod \"multus-additional-cni-plugins-b9mxn\" (UID: \"36c66dd3-6c06-4883-8ccd-107c566a45d6\") " pod="openshift-multus/multus-additional-cni-plugins-b9mxn" Jul 28 17:08:23.813789 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.813352 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-host-var-lib-kubelet\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.813789 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.813387 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dllpj\" (UniqueName: \"kubernetes.io/projected/75339f13-4483-4c0f-a502-885f0b22c8b7-kube-api-access-dllpj\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.813789 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.813421 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd54c89e-8c35-404b-8804-ca3e45d384b4-tmp-dir\") pod \"node-resolver-zbmdl\" (UID: \"dd54c89e-8c35-404b-8804-ca3e45d384b4\") " pod="openshift-dns/node-resolver-zbmdl" Jul 28 17:08:23.813789 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.813448 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/edd0a7b9-158c-4fdf-8069-5e49e0ce0772-registration-dir\") pod \"aws-ebs-csi-driver-node-ksszp\" (UID: \"edd0a7b9-158c-4fdf-8069-5e49e0ce0772\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksszp" Jul 28 17:08:23.813789 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.813480 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rl5j\" (UniqueName: \"kubernetes.io/projected/93b6b3b5-e860-4fee-9800-0f4534b72e67-kube-api-access-9rl5j\") pod \"network-metrics-daemon-5tdlw\" (UID: \"93b6b3b5-e860-4fee-9800-0f4534b72e67\") " pod="openshift-multus/network-metrics-daemon-5tdlw" Jul 28 17:08:23.813789 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.813520 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffxdp\" (UniqueName: \"kubernetes.io/projected/56d1a325-976b-4060-b1a4-1a4b6be0bd18-kube-api-access-ffxdp\") pod \"node-ca-cmx6g\" (UID: \"56d1a325-976b-4060-b1a4-1a4b6be0bd18\") " pod="openshift-image-registry/node-ca-cmx6g" Jul 28 17:08:23.813789 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.813608 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-host-run-multus-certs\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.813789 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.813737 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-cnibin\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.813789 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.813789 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/36c66dd3-6c06-4883-8ccd-107c566a45d6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b9mxn\" (UID: \"36c66dd3-6c06-4883-8ccd-107c566a45d6\") " pod="openshift-multus/multus-additional-cni-plugins-b9mxn" Jul 28 17:08:23.814401 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.813874 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnhxc\" (UniqueName: \"kubernetes.io/projected/36c66dd3-6c06-4883-8ccd-107c566a45d6-kube-api-access-pnhxc\") pod \"multus-additional-cni-plugins-b9mxn\" (UID: \"36c66dd3-6c06-4883-8ccd-107c566a45d6\") " pod="openshift-multus/multus-additional-cni-plugins-b9mxn" Jul 28 17:08:23.814401 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.814053 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-multus-socket-dir-parent\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.814401 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.814108 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/edd0a7b9-158c-4fdf-8069-5e49e0ce0772-socket-dir\") pod \"aws-ebs-csi-driver-node-ksszp\" (UID: \"edd0a7b9-158c-4fdf-8069-5e49e0ce0772\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksszp" Jul 28 17:08:23.814401 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.814199 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/edd0a7b9-158c-4fdf-8069-5e49e0ce0772-device-dir\") pod \"aws-ebs-csi-driver-node-ksszp\" (UID: \"edd0a7b9-158c-4fdf-8069-5e49e0ce0772\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksszp" Jul 28 17:08:23.814401 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.814242 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-host-run-k8s-cni-cncf-io\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.814703 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.814443 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/edd0a7b9-158c-4fdf-8069-5e49e0ce0772-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ksszp\" (UID: \"edd0a7b9-158c-4fdf-8069-5e49e0ce0772\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksszp" Jul 28 17:08:23.814703 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.814481 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93b6b3b5-e860-4fee-9800-0f4534b72e67-metrics-certs\") pod \"network-metrics-daemon-5tdlw\" (UID: \"93b6b3b5-e860-4fee-9800-0f4534b72e67\") " pod="openshift-multus/network-metrics-daemon-5tdlw" Jul 28 17:08:23.814703 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.814617 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 28 17:08:23.814703 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.814646 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:23.814848 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.814603 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-system-cni-dir\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.816126 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.816105 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-wnq5m\"" Jul 28 17:08:23.817207 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.816490 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 28 17:08:23.818557 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.818519 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dq76s\"" Jul 28 17:08:23.818666 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.818548 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 28 17:08:23.818666 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.818608 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 28 17:08:23.818784 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.818763 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 28 17:08:23.820008 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.819977 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vmdd2" Jul 28 17:08:23.821403 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.821381 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 28 17:08:23.821575 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.821546 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 28 17:08:23.822608 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.822590 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4zv65\"" Jul 28 17:08:23.822955 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.822931 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 28 17:08:23.823040 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.822930 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 28 17:08:23.823321 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.823299 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 28 17:08:23.823999 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.823971 2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 28 17:08:23.825017 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.824998 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 28 17:08:23.847873 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.847852 2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-l45f2" Jul 28 17:08:23.859833 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.859815 2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-l45f2" Jul 28 17:08:23.901533 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.901506 2570 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 28 17:08:23.915780 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.915754 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-run-systemd\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:23.915897 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.915800 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/36c66dd3-6c06-4883-8ccd-107c566a45d6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b9mxn\" (UID: \"36c66dd3-6c06-4883-8ccd-107c566a45d6\") " pod="openshift-multus/multus-additional-cni-plugins-b9mxn" Jul 28 17:08:23.915897 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.915831 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/75339f13-4483-4c0f-a502-885f0b22c8b7-cni-binary-copy\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.915897 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.915858 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nn58s\" (UniqueName: \"kubernetes.io/projected/edd0a7b9-158c-4fdf-8069-5e49e0ce0772-kube-api-access-nn58s\") pod \"aws-ebs-csi-driver-node-ksszp\" (UID: \"edd0a7b9-158c-4fdf-8069-5e49e0ce0772\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksszp" Jul 28 17:08:23.915897 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.915880 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffxdp\" (UniqueName: \"kubernetes.io/projected/56d1a325-976b-4060-b1a4-1a4b6be0bd18-kube-api-access-ffxdp\") pod \"node-ca-cmx6g\" (UID: \"56d1a325-976b-4060-b1a4-1a4b6be0bd18\") " pod="openshift-image-registry/node-ca-cmx6g" Jul 28 17:08:23.916130 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.915896 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e2d09d1f-c076-4633-aa92-acb1adc2cbfe-konnectivity-ca\") pod \"konnectivity-agent-dmc56\" (UID: \"e2d09d1f-c076-4633-aa92-acb1adc2cbfe\") " pod="kube-system/konnectivity-agent-dmc56" Jul 28 17:08:23.916130 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.915926 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-host-kubelet\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:23.916130 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.915950 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-host-var-lib-kubelet\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.916130 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.915966 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dllpj\" (UniqueName: \"kubernetes.io/projected/75339f13-4483-4c0f-a502-885f0b22c8b7-kube-api-access-dllpj\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.916130 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.915984 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/edd0a7b9-158c-4fdf-8069-5e49e0ce0772-registration-dir\") pod \"aws-ebs-csi-driver-node-ksszp\" (UID: \"edd0a7b9-158c-4fdf-8069-5e49e0ce0772\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksszp" Jul 28 17:08:23.916130 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916004 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9baa058e-a765-4c65-b5a6-ce0a7c022d97-original-pull-secret\") pod \"global-pull-secret-syncer-fbzsz\" (UID: \"9baa058e-a765-4c65-b5a6-ce0a7c022d97\") " pod="kube-system/global-pull-secret-syncer-fbzsz" Jul 28 17:08:23.916130 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916028 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-host-run-multus-certs\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.916130 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916045 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/36c66dd3-6c06-4883-8ccd-107c566a45d6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b9mxn\" (UID: \"36c66dd3-6c06-4883-8ccd-107c566a45d6\") " pod="openshift-multus/multus-additional-cni-plugins-b9mxn" Jul 28 17:08:23.916130 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916065 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/edd0a7b9-158c-4fdf-8069-5e49e0ce0772-socket-dir\") pod \"aws-ebs-csi-driver-node-ksszp\" (UID: \"edd0a7b9-158c-4fdf-8069-5e49e0ce0772\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksszp" Jul 28 17:08:23.916130 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916113 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b71d4a48-0ead-41ac-be1b-ce910838865b-etc-sysctl-conf\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:23.916701 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916175 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-var-lib-openvswitch\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:23.916701 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916201 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-node-log\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:23.916701 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916243 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-host-run-k8s-cni-cncf-io\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.916701 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916283 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/edd0a7b9-158c-4fdf-8069-5e49e0ce0772-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ksszp\" (UID: \"edd0a7b9-158c-4fdf-8069-5e49e0ce0772\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksszp" Jul 28 17:08:23.916701 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916303 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2972842b-5159-4e7e-8330-da3425927ea0-iptables-alerter-script\") pod \"iptables-alerter-vmdd2\" (UID: \"2972842b-5159-4e7e-8330-da3425927ea0\") " pod="openshift-network-operator/iptables-alerter-vmdd2" Jul 28 17:08:23.916701 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916327 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-host-slash\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:23.916701 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916344 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-log-socket\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:23.916701 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916361 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9v8gx\" (UniqueName: \"kubernetes.io/projected/c341008d-4e07-448d-9630-8d9a348088af-kube-api-access-9v8gx\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:23.916701 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916377 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-multus-cni-dir\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.916701 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916400 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/56d1a325-976b-4060-b1a4-1a4b6be0bd18-host\") pod \"node-ca-cmx6g\" (UID: \"56d1a325-976b-4060-b1a4-1a4b6be0bd18\") " pod="openshift-image-registry/node-ca-cmx6g" Jul 28 17:08:23.916701 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916423 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/56d1a325-976b-4060-b1a4-1a4b6be0bd18-serviceca\") pod \"node-ca-cmx6g\" (UID: \"56d1a325-976b-4060-b1a4-1a4b6be0bd18\") " pod="openshift-image-registry/node-ca-cmx6g" Jul 28 17:08:23.916701 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916444 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e2d09d1f-c076-4633-aa92-acb1adc2cbfe-agent-certs\") pod \"konnectivity-agent-dmc56\" (UID: \"e2d09d1f-c076-4633-aa92-acb1adc2cbfe\") " pod="kube-system/konnectivity-agent-dmc56" Jul 28 17:08:23.916701 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916460 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b71d4a48-0ead-41ac-be1b-ce910838865b-sys\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:23.916701 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916479 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b71d4a48-0ead-41ac-be1b-ce910838865b-host\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:23.916701 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916508 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:23.916701 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916531 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-os-release\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.916701 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916557 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lzzz\" (UniqueName: \"kubernetes.io/projected/980a4184-ed46-42a1-99f3-86c49cfba43a-kube-api-access-7lzzz\") pod \"network-check-target-6dbsb\" (UID: \"980a4184-ed46-42a1-99f3-86c49cfba43a\") " pod="openshift-network-diagnostics/network-check-target-6dbsb" Jul 28 17:08:23.917558 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916587 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b71d4a48-0ead-41ac-be1b-ce910838865b-lib-modules\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:23.917558 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916613 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-run-openvswitch\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:23.917558 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916638 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-host-cni-netd\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:23.917558 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916661 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-host-var-lib-cni-bin\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.917558 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916679 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-host-var-lib-cni-multus\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.917558 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916700 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-multus-conf-dir\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.917558 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916716 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/75339f13-4483-4c0f-a502-885f0b22c8b7-multus-daemon-config\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.917558 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916733 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-etc-kubernetes\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.917558 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916751 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-host-run-netns\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.917558 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916773 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-hostroot\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.917558 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916791 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dd54c89e-8c35-404b-8804-ca3e45d384b4-hosts-file\") pod \"node-resolver-zbmdl\" (UID: \"dd54c89e-8c35-404b-8804-ca3e45d384b4\") " pod="openshift-dns/node-resolver-zbmdl" Jul 28 17:08:23.917558 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916815 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/edd0a7b9-158c-4fdf-8069-5e49e0ce0772-etc-selinux\") pod \"aws-ebs-csi-driver-node-ksszp\" (UID: \"edd0a7b9-158c-4fdf-8069-5e49e0ce0772\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksszp" Jul 28 17:08:23.917558 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916831 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b71d4a48-0ead-41ac-be1b-ce910838865b-var-lib-kubelet\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:23.917558 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916848 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-host-run-ovn-kubernetes\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:23.917558 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916867 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/36c66dd3-6c06-4883-8ccd-107c566a45d6-cni-binary-copy\") pod \"multus-additional-cni-plugins-b9mxn\" (UID: \"36c66dd3-6c06-4883-8ccd-107c566a45d6\") " pod="openshift-multus/multus-additional-cni-plugins-b9mxn" Jul 28 17:08:23.917558 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916884 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd54c89e-8c35-404b-8804-ca3e45d384b4-tmp-dir\") pod \"node-resolver-zbmdl\" (UID: \"dd54c89e-8c35-404b-8804-ca3e45d384b4\") " pod="openshift-dns/node-resolver-zbmdl" Jul 28 17:08:23.917558 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916915 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rl5j\" (UniqueName: \"kubernetes.io/projected/93b6b3b5-e860-4fee-9800-0f4534b72e67-kube-api-access-9rl5j\") pod \"network-metrics-daemon-5tdlw\" (UID: \"93b6b3b5-e860-4fee-9800-0f4534b72e67\") " pod="openshift-multus/network-metrics-daemon-5tdlw" Jul 28 17:08:23.918403 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916932 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tg9b\" (UniqueName: \"kubernetes.io/projected/b71d4a48-0ead-41ac-be1b-ce910838865b-kube-api-access-9tg9b\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:23.918403 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916954 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/36c66dd3-6c06-4883-8ccd-107c566a45d6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b9mxn\" (UID: \"36c66dd3-6c06-4883-8ccd-107c566a45d6\") " pod="openshift-multus/multus-additional-cni-plugins-b9mxn" Jul 28 17:08:23.918403 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916965 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-host-cni-bin\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:23.918403 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916982 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-cnibin\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.918403 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916999 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnhxc\" (UniqueName: \"kubernetes.io/projected/36c66dd3-6c06-4883-8ccd-107c566a45d6-kube-api-access-pnhxc\") pod \"multus-additional-cni-plugins-b9mxn\" (UID: \"36c66dd3-6c06-4883-8ccd-107c566a45d6\") " pod="openshift-multus/multus-additional-cni-plugins-b9mxn" Jul 28 17:08:23.918403 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.917037 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-multus-socket-dir-parent\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.918403 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.917058 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/edd0a7b9-158c-4fdf-8069-5e49e0ce0772-device-dir\") pod \"aws-ebs-csi-driver-node-ksszp\" (UID: \"edd0a7b9-158c-4fdf-8069-5e49e0ce0772\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksszp" Jul 28 17:08:23.918403 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.917074 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9baa058e-a765-4c65-b5a6-ce0a7c022d97-dbus\") pod \"global-pull-secret-syncer-fbzsz\" (UID: \"9baa058e-a765-4c65-b5a6-ce0a7c022d97\") " pod="kube-system/global-pull-secret-syncer-fbzsz" Jul 28 17:08:23.918403 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.917091 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b71d4a48-0ead-41ac-be1b-ce910838865b-etc-modprobe-d\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:23.918403 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.917097 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/36c66dd3-6c06-4883-8ccd-107c566a45d6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b9mxn\" (UID: \"36c66dd3-6c06-4883-8ccd-107c566a45d6\") " pod="openshift-multus/multus-additional-cni-plugins-b9mxn" Jul 28 17:08:23.918403 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.917110 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b71d4a48-0ead-41ac-be1b-ce910838865b-etc-sysctl-d\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:23.918403 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.917132 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b71d4a48-0ead-41ac-be1b-ce910838865b-run\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:23.918403 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.917149 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93b6b3b5-e860-4fee-9800-0f4534b72e67-metrics-certs\") pod \"network-metrics-daemon-5tdlw\" (UID: \"93b6b3b5-e860-4fee-9800-0f4534b72e67\") " pod="openshift-multus/network-metrics-daemon-5tdlw" Jul 28 17:08:23.918403 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.917167 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9baa058e-a765-4c65-b5a6-ce0a7c022d97-kubelet-config\") pod \"global-pull-secret-syncer-fbzsz\" (UID: \"9baa058e-a765-4c65-b5a6-ce0a7c022d97\") " pod="kube-system/global-pull-secret-syncer-fbzsz" Jul 28 17:08:23.918403 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.917183 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c341008d-4e07-448d-9630-8d9a348088af-ovnkube-config\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:23.918403 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.917205 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/edd0a7b9-158c-4fdf-8069-5e49e0ce0772-registration-dir\") pod \"aws-ebs-csi-driver-node-ksszp\" (UID: \"edd0a7b9-158c-4fdf-8069-5e49e0ce0772\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksszp" Jul 28 17:08:23.918403 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.917218 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c341008d-4e07-448d-9630-8d9a348088af-env-overrides\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:23.919286 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.917248 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c341008d-4e07-448d-9630-8d9a348088af-ovn-node-metrics-cert\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:23.919286 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.917272 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-host-var-lib-kubelet\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.919286 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.917292 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/edd0a7b9-158c-4fdf-8069-5e49e0ce0772-socket-dir\") pod \"aws-ebs-csi-driver-node-ksszp\" (UID: \"edd0a7b9-158c-4fdf-8069-5e49e0ce0772\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksszp" Jul 28 17:08:23.919286 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.917313 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-host-run-k8s-cni-cncf-io\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.919286 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.916478 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-host-run-multus-certs\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.919286 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.917341 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/edd0a7b9-158c-4fdf-8069-5e49e0ce0772-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ksszp\" (UID: \"edd0a7b9-158c-4fdf-8069-5e49e0ce0772\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksszp" Jul 28 17:08:23.919286 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.917398 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/56d1a325-976b-4060-b1a4-1a4b6be0bd18-host\") pod \"node-ca-cmx6g\" (UID: \"56d1a325-976b-4060-b1a4-1a4b6be0bd18\") " pod="openshift-image-registry/node-ca-cmx6g" Jul 28 17:08:23.919286 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.917417 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/75339f13-4483-4c0f-a502-885f0b22c8b7-cni-binary-copy\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.919286 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.917482 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-system-cni-dir\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.919286 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.917507 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-cnibin\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.919286 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:23.917514 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:08:23.919286 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.917550 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/36c66dd3-6c06-4883-8ccd-107c566a45d6-system-cni-dir\") pod \"multus-additional-cni-plugins-b9mxn\" (UID: \"36c66dd3-6c06-4883-8ccd-107c566a45d6\") " pod="openshift-multus/multus-additional-cni-plugins-b9mxn" Jul 28 17:08:23.919286 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.917647 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd54c89e-8c35-404b-8804-ca3e45d384b4-tmp-dir\") pod \"node-resolver-zbmdl\" (UID: \"dd54c89e-8c35-404b-8804-ca3e45d384b4\") " pod="openshift-dns/node-resolver-zbmdl" Jul 28 17:08:23.919286 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.917670 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-multus-socket-dir-parent\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.919286 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.917748 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/edd0a7b9-158c-4fdf-8069-5e49e0ce0772-device-dir\") pod \"aws-ebs-csi-driver-node-ksszp\" (UID: \"edd0a7b9-158c-4fdf-8069-5e49e0ce0772\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksszp" Jul 28 17:08:23.919286 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:23.917810 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93b6b3b5-e860-4fee-9800-0f4534b72e67-metrics-certs podName:93b6b3b5-e860-4fee-9800-0f4534b72e67 nodeName:}" failed. No retries permitted until 2026-07-28 17:08:24.417782908 +0000 UTC m=+3.026714276 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/93b6b3b5-e860-4fee-9800-0f4534b72e67-metrics-certs") pod "network-metrics-daemon-5tdlw" (UID: "93b6b3b5-e860-4fee-9800-0f4534b72e67") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:08:23.919286 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.917821 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-system-cni-dir\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.919890 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.917860 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/36c66dd3-6c06-4883-8ccd-107c566a45d6-cni-binary-copy\") pod \"multus-additional-cni-plugins-b9mxn\" (UID: \"36c66dd3-6c06-4883-8ccd-107c566a45d6\") " pod="openshift-multus/multus-additional-cni-plugins-b9mxn" Jul 28 17:08:23.919890 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.917892 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-host-var-lib-cni-bin\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.919890 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.917901 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-host-run-netns\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.919890 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.917979 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-os-release\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.919890 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.917968 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-multus-cni-dir\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.919890 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.917516 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/36c66dd3-6c06-4883-8ccd-107c566a45d6-system-cni-dir\") pod \"multus-additional-cni-plugins-b9mxn\" (UID: \"36c66dd3-6c06-4883-8ccd-107c566a45d6\") " pod="openshift-multus/multus-additional-cni-plugins-b9mxn" Jul 28 17:08:23.919890 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.918050 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-host-var-lib-cni-multus\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.919890 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.918087 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/36c66dd3-6c06-4883-8ccd-107c566a45d6-cnibin\") pod \"multus-additional-cni-plugins-b9mxn\" (UID: \"36c66dd3-6c06-4883-8ccd-107c566a45d6\") " pod="openshift-multus/multus-additional-cni-plugins-b9mxn" Jul 28 17:08:23.919890 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.918118 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-etc-kubernetes\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.919890 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.918130 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b71d4a48-0ead-41ac-be1b-ce910838865b-tmp\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:23.919890 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.918169 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-etc-openvswitch\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:23.919890 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.918223 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-run-ovn\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:23.919890 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.918261 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c341008d-4e07-448d-9630-8d9a348088af-ovnkube-script-lib\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:23.919890 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.918276 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-hostroot\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.919890 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.918294 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/36c66dd3-6c06-4883-8ccd-107c566a45d6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b9mxn\" (UID: \"36c66dd3-6c06-4883-8ccd-107c566a45d6\") " pod="openshift-multus/multus-additional-cni-plugins-b9mxn" Jul 28 17:08:23.919890 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.918367 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/56d1a325-976b-4060-b1a4-1a4b6be0bd18-serviceca\") pod \"node-ca-cmx6g\" (UID: \"56d1a325-976b-4060-b1a4-1a4b6be0bd18\") " pod="openshift-image-registry/node-ca-cmx6g" Jul 28 17:08:23.919890 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.918393 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/edd0a7b9-158c-4fdf-8069-5e49e0ce0772-etc-selinux\") pod \"aws-ebs-csi-driver-node-ksszp\" (UID: \"edd0a7b9-158c-4fdf-8069-5e49e0ce0772\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksszp" Jul 28 17:08:23.920804 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.918416 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/36c66dd3-6c06-4883-8ccd-107c566a45d6-cnibin\") pod \"multus-additional-cni-plugins-b9mxn\" (UID: \"36c66dd3-6c06-4883-8ccd-107c566a45d6\") " pod="openshift-multus/multus-additional-cni-plugins-b9mxn" Jul 28 17:08:23.920804 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.918416 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dd54c89e-8c35-404b-8804-ca3e45d384b4-hosts-file\") pod \"node-resolver-zbmdl\" (UID: \"dd54c89e-8c35-404b-8804-ca3e45d384b4\") " pod="openshift-dns/node-resolver-zbmdl" Jul 28 17:08:23.920804 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.918480 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/75339f13-4483-4c0f-a502-885f0b22c8b7-multus-conf-dir\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.920804 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.918508 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nz59s\" (UniqueName: \"kubernetes.io/projected/2972842b-5159-4e7e-8330-da3425927ea0-kube-api-access-nz59s\") pod \"iptables-alerter-vmdd2\" (UID: \"2972842b-5159-4e7e-8330-da3425927ea0\") " pod="openshift-network-operator/iptables-alerter-vmdd2" Jul 28 17:08:23.920804 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.918539 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/36c66dd3-6c06-4883-8ccd-107c566a45d6-os-release\") pod \"multus-additional-cni-plugins-b9mxn\" (UID: \"36c66dd3-6c06-4883-8ccd-107c566a45d6\") " pod="openshift-multus/multus-additional-cni-plugins-b9mxn" Jul 28 17:08:23.920804 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.918561 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgvp5\" (UniqueName: \"kubernetes.io/projected/dd54c89e-8c35-404b-8804-ca3e45d384b4-kube-api-access-xgvp5\") pod \"node-resolver-zbmdl\" (UID: \"dd54c89e-8c35-404b-8804-ca3e45d384b4\") " pod="openshift-dns/node-resolver-zbmdl" Jul 28 17:08:23.920804 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.918598 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/edd0a7b9-158c-4fdf-8069-5e49e0ce0772-sys-fs\") pod \"aws-ebs-csi-driver-node-ksszp\" (UID: \"edd0a7b9-158c-4fdf-8069-5e49e0ce0772\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksszp" Jul 28 17:08:23.920804 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.918632 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/36c66dd3-6c06-4883-8ccd-107c566a45d6-os-release\") pod \"multus-additional-cni-plugins-b9mxn\" (UID: \"36c66dd3-6c06-4883-8ccd-107c566a45d6\") " pod="openshift-multus/multus-additional-cni-plugins-b9mxn" Jul 28 17:08:23.920804 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.918694 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b71d4a48-0ead-41ac-be1b-ce910838865b-etc-systemd\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:23.920804 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.918699 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/75339f13-4483-4c0f-a502-885f0b22c8b7-multus-daemon-config\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.920804 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.918726 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2972842b-5159-4e7e-8330-da3425927ea0-host-slash\") pod \"iptables-alerter-vmdd2\" (UID: \"2972842b-5159-4e7e-8330-da3425927ea0\") " pod="openshift-network-operator/iptables-alerter-vmdd2" Jul 28 17:08:23.920804 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.918788 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/edd0a7b9-158c-4fdf-8069-5e49e0ce0772-sys-fs\") pod \"aws-ebs-csi-driver-node-ksszp\" (UID: \"edd0a7b9-158c-4fdf-8069-5e49e0ce0772\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksszp" Jul 28 17:08:23.920804 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.918791 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b71d4a48-0ead-41ac-be1b-ce910838865b-etc-sysconfig\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:23.920804 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.918838 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b71d4a48-0ead-41ac-be1b-ce910838865b-etc-kubernetes\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:23.920804 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.918877 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b71d4a48-0ead-41ac-be1b-ce910838865b-etc-tuned\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:23.920804 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.918877 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/36c66dd3-6c06-4883-8ccd-107c566a45d6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b9mxn\" (UID: \"36c66dd3-6c06-4883-8ccd-107c566a45d6\") " pod="openshift-multus/multus-additional-cni-plugins-b9mxn" Jul 28 17:08:23.920804 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.918902 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-systemd-units\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:23.921598 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.918956 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-host-run-netns\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:23.928514 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:23.928419 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:08:23.928514 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:23.928442 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:08:23.928514 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:23.928457 2570 projected.go:196] Error preparing data for projected volume kube-api-access-7lzzz for pod openshift-network-diagnostics/network-check-target-6dbsb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:08:23.928514 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:23.928515 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/980a4184-ed46-42a1-99f3-86c49cfba43a-kube-api-access-7lzzz podName:980a4184-ed46-42a1-99f3-86c49cfba43a nodeName:}" failed. No retries permitted until 2026-07-28 17:08:24.428498095 +0000 UTC m=+3.037429461 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-7lzzz" (UniqueName: "kubernetes.io/projected/980a4184-ed46-42a1-99f3-86c49cfba43a-kube-api-access-7lzzz") pod "network-check-target-6dbsb" (UID: "980a4184-ed46-42a1-99f3-86c49cfba43a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:08:23.928855 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.928678 2570 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 28 17:08:23.933151 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.933127 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dllpj\" (UniqueName: \"kubernetes.io/projected/75339f13-4483-4c0f-a502-885f0b22c8b7-kube-api-access-dllpj\") pod \"multus-jm8ng\" (UID: \"75339f13-4483-4c0f-a502-885f0b22c8b7\") " pod="openshift-multus/multus-jm8ng" Jul 28 17:08:23.933259 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.933129 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnhxc\" (UniqueName: \"kubernetes.io/projected/36c66dd3-6c06-4883-8ccd-107c566a45d6-kube-api-access-pnhxc\") pod \"multus-additional-cni-plugins-b9mxn\" (UID: \"36c66dd3-6c06-4883-8ccd-107c566a45d6\") " pod="openshift-multus/multus-additional-cni-plugins-b9mxn" Jul 28 17:08:23.933605 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.933580 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rl5j\" (UniqueName: \"kubernetes.io/projected/93b6b3b5-e860-4fee-9800-0f4534b72e67-kube-api-access-9rl5j\") pod \"network-metrics-daemon-5tdlw\" (UID: \"93b6b3b5-e860-4fee-9800-0f4534b72e67\") " pod="openshift-multus/network-metrics-daemon-5tdlw" Jul 28 17:08:23.935138 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.935091 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nn58s\" (UniqueName: \"kubernetes.io/projected/edd0a7b9-158c-4fdf-8069-5e49e0ce0772-kube-api-access-nn58s\") pod \"aws-ebs-csi-driver-node-ksszp\" (UID: \"edd0a7b9-158c-4fdf-8069-5e49e0ce0772\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksszp" Jul 28 17:08:23.936628 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.936590 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffxdp\" (UniqueName: \"kubernetes.io/projected/56d1a325-976b-4060-b1a4-1a4b6be0bd18-kube-api-access-ffxdp\") pod \"node-ca-cmx6g\" (UID: \"56d1a325-976b-4060-b1a4-1a4b6be0bd18\") " pod="openshift-image-registry/node-ca-cmx6g" Jul 28 17:08:23.937425 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:23.936778 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgvp5\" (UniqueName: \"kubernetes.io/projected/dd54c89e-8c35-404b-8804-ca3e45d384b4-kube-api-access-xgvp5\") pod \"node-resolver-zbmdl\" (UID: \"dd54c89e-8c35-404b-8804-ca3e45d384b4\") " pod="openshift-dns/node-resolver-zbmdl" Jul 28 17:08:24.020105 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.020064 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b71d4a48-0ead-41ac-be1b-ce910838865b-tmp\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:24.020105 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.020115 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-etc-openvswitch\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.020364 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.020156 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-run-ovn\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.020364 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.020182 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c341008d-4e07-448d-9630-8d9a348088af-ovnkube-script-lib\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.020364 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.020210 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz59s\" (UniqueName: \"kubernetes.io/projected/2972842b-5159-4e7e-8330-da3425927ea0-kube-api-access-nz59s\") pod \"iptables-alerter-vmdd2\" (UID: \"2972842b-5159-4e7e-8330-da3425927ea0\") " pod="openshift-network-operator/iptables-alerter-vmdd2" Jul 28 17:08:24.020364 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.020239 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b71d4a48-0ead-41ac-be1b-ce910838865b-etc-systemd\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:24.020364 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.020262 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2972842b-5159-4e7e-8330-da3425927ea0-host-slash\") pod \"iptables-alerter-vmdd2\" (UID: \"2972842b-5159-4e7e-8330-da3425927ea0\") " pod="openshift-network-operator/iptables-alerter-vmdd2" Jul 28 17:08:24.020364 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.020285 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b71d4a48-0ead-41ac-be1b-ce910838865b-etc-sysconfig\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:24.020364 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.020308 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b71d4a48-0ead-41ac-be1b-ce910838865b-etc-kubernetes\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:24.020364 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.020329 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b71d4a48-0ead-41ac-be1b-ce910838865b-etc-tuned\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:24.020364 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.020362 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-systemd-units\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.020877 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.020393 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-host-run-netns\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.020877 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.020418 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-run-systemd\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.020877 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.020448 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e2d09d1f-c076-4633-aa92-acb1adc2cbfe-konnectivity-ca\") pod \"konnectivity-agent-dmc56\" (UID: \"e2d09d1f-c076-4633-aa92-acb1adc2cbfe\") " pod="kube-system/konnectivity-agent-dmc56" Jul 28 17:08:24.020877 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.020471 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-host-kubelet\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.020877 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.020500 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9baa058e-a765-4c65-b5a6-ce0a7c022d97-original-pull-secret\") pod \"global-pull-secret-syncer-fbzsz\" (UID: \"9baa058e-a765-4c65-b5a6-ce0a7c022d97\") " pod="kube-system/global-pull-secret-syncer-fbzsz" Jul 28 17:08:24.020877 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.020529 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b71d4a48-0ead-41ac-be1b-ce910838865b-etc-sysctl-conf\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:24.020877 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.020553 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-var-lib-openvswitch\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.020877 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.020581 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-node-log\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.020877 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.020608 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2972842b-5159-4e7e-8330-da3425927ea0-iptables-alerter-script\") pod \"iptables-alerter-vmdd2\" (UID: \"2972842b-5159-4e7e-8330-da3425927ea0\") " pod="openshift-network-operator/iptables-alerter-vmdd2" Jul 28 17:08:24.020877 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.020631 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-host-slash\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.020877 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.020657 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-log-socket\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.020877 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.020709 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-node-log\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.020877 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.020714 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-host-kubelet\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.020877 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.020768 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v8gx\" (UniqueName: \"kubernetes.io/projected/c341008d-4e07-448d-9630-8d9a348088af-kube-api-access-9v8gx\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.020877 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.020871 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e2d09d1f-c076-4633-aa92-acb1adc2cbfe-agent-certs\") pod \"konnectivity-agent-dmc56\" (UID: \"e2d09d1f-c076-4633-aa92-acb1adc2cbfe\") " pod="kube-system/konnectivity-agent-dmc56" Jul 28 17:08:24.020877 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.020872 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b71d4a48-0ead-41ac-be1b-ce910838865b-etc-sysctl-conf\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:24.021700 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.020921 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b71d4a48-0ead-41ac-be1b-ce910838865b-sys\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:24.021700 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.020952 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b71d4a48-0ead-41ac-be1b-ce910838865b-host\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:24.021700 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.020981 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.021700 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.021028 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b71d4a48-0ead-41ac-be1b-ce910838865b-lib-modules\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:24.021700 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.021055 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-run-openvswitch\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.021700 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.021081 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-host-cni-netd\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.021700 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.021120 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b71d4a48-0ead-41ac-be1b-ce910838865b-var-lib-kubelet\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:24.021700 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.021146 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-host-run-ovn-kubernetes\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.021700 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.021176 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tg9b\" (UniqueName: \"kubernetes.io/projected/b71d4a48-0ead-41ac-be1b-ce910838865b-kube-api-access-9tg9b\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:24.021700 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.021219 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-host-cni-bin\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.021700 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.021250 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9baa058e-a765-4c65-b5a6-ce0a7c022d97-dbus\") pod \"global-pull-secret-syncer-fbzsz\" (UID: \"9baa058e-a765-4c65-b5a6-ce0a7c022d97\") " pod="kube-system/global-pull-secret-syncer-fbzsz" Jul 28 17:08:24.021700 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.021276 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b71d4a48-0ead-41ac-be1b-ce910838865b-etc-modprobe-d\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:24.021700 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.021304 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b71d4a48-0ead-41ac-be1b-ce910838865b-etc-sysctl-d\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:24.021700 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.021319 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2972842b-5159-4e7e-8330-da3425927ea0-iptables-alerter-script\") pod \"iptables-alerter-vmdd2\" (UID: \"2972842b-5159-4e7e-8330-da3425927ea0\") " pod="openshift-network-operator/iptables-alerter-vmdd2" Jul 28 17:08:24.021700 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.021327 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b71d4a48-0ead-41ac-be1b-ce910838865b-run\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:24.021700 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.021329 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c341008d-4e07-448d-9630-8d9a348088af-ovnkube-script-lib\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.021700 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.021379 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-etc-openvswitch\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.022582 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.021393 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-log-socket\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.022582 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:24.021403 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:08:24.022582 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.021454 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b71d4a48-0ead-41ac-be1b-ce910838865b-sys\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:24.022582 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:24.021469 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9baa058e-a765-4c65-b5a6-ce0a7c022d97-original-pull-secret podName:9baa058e-a765-4c65-b5a6-ce0a7c022d97 nodeName:}" failed. No retries permitted until 2026-07-28 17:08:24.521452289 +0000 UTC m=+3.130383656 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9baa058e-a765-4c65-b5a6-ce0a7c022d97-original-pull-secret") pod "global-pull-secret-syncer-fbzsz" (UID: "9baa058e-a765-4c65-b5a6-ce0a7c022d97") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:08:24.022582 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.021512 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-host-slash\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.022582 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.021513 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b71d4a48-0ead-41ac-be1b-ce910838865b-host\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:24.022582 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.020708 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-run-ovn\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.022582 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.021551 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.022582 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.021562 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9baa058e-a765-4c65-b5a6-ce0a7c022d97-kubelet-config\") pod \"global-pull-secret-syncer-fbzsz\" (UID: \"9baa058e-a765-4c65-b5a6-ce0a7c022d97\") " pod="kube-system/global-pull-secret-syncer-fbzsz" Jul 28 17:08:24.022582 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.021605 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c341008d-4e07-448d-9630-8d9a348088af-ovnkube-config\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.022582 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.021631 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c341008d-4e07-448d-9630-8d9a348088af-env-overrides\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.022582 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.021642 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b71d4a48-0ead-41ac-be1b-ce910838865b-etc-systemd\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:24.022582 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.021658 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c341008d-4e07-448d-9630-8d9a348088af-ovn-node-metrics-cert\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.022582 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.021690 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2972842b-5159-4e7e-8330-da3425927ea0-host-slash\") pod \"iptables-alerter-vmdd2\" (UID: \"2972842b-5159-4e7e-8330-da3425927ea0\") " pod="openshift-network-operator/iptables-alerter-vmdd2" Jul 28 17:08:24.022582 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.021740 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b71d4a48-0ead-41ac-be1b-ce910838865b-etc-sysconfig\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:24.022582 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.021792 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b71d4a48-0ead-41ac-be1b-ce910838865b-etc-kubernetes\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:24.022582 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.021798 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-host-cni-bin\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.023533 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.021849 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b71d4a48-0ead-41ac-be1b-ce910838865b-run\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:24.023533 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.021897 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9baa058e-a765-4c65-b5a6-ce0a7c022d97-dbus\") pod \"global-pull-secret-syncer-fbzsz\" (UID: \"9baa058e-a765-4c65-b5a6-ce0a7c022d97\") " pod="kube-system/global-pull-secret-syncer-fbzsz" Jul 28 17:08:24.023533 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.021936 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b71d4a48-0ead-41ac-be1b-ce910838865b-lib-modules\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:24.023533 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.021977 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b71d4a48-0ead-41ac-be1b-ce910838865b-etc-modprobe-d\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:24.023533 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.021983 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-run-openvswitch\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.023533 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.022022 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-host-cni-netd\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.023533 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.022072 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b71d4a48-0ead-41ac-be1b-ce910838865b-etc-sysctl-d\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:24.023533 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.022091 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b71d4a48-0ead-41ac-be1b-ce910838865b-var-lib-kubelet\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:24.023533 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.021606 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9baa058e-a765-4c65-b5a6-ce0a7c022d97-kubelet-config\") pod \"global-pull-secret-syncer-fbzsz\" (UID: \"9baa058e-a765-4c65-b5a6-ce0a7c022d97\") " pod="kube-system/global-pull-secret-syncer-fbzsz" Jul 28 17:08:24.023533 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.022118 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-host-run-ovn-kubernetes\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.023533 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.022280 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-var-lib-openvswitch\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.023533 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.022336 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-host-run-netns\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.023533 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.022386 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-systemd-units\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.023533 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.022427 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c341008d-4e07-448d-9630-8d9a348088af-run-systemd\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.023533 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.022782 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c341008d-4e07-448d-9630-8d9a348088af-env-overrides\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.023533 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.023215 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c341008d-4e07-448d-9630-8d9a348088af-ovnkube-config\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.023533 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.023340 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e2d09d1f-c076-4633-aa92-acb1adc2cbfe-konnectivity-ca\") pod \"konnectivity-agent-dmc56\" (UID: \"e2d09d1f-c076-4633-aa92-acb1adc2cbfe\") " pod="kube-system/konnectivity-agent-dmc56" Jul 28 17:08:24.025145 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.025098 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b71d4a48-0ead-41ac-be1b-ce910838865b-tmp\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:24.025724 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.025676 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b71d4a48-0ead-41ac-be1b-ce910838865b-etc-tuned\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:24.025835 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.025770 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e2d09d1f-c076-4633-aa92-acb1adc2cbfe-agent-certs\") pod \"konnectivity-agent-dmc56\" (UID: \"e2d09d1f-c076-4633-aa92-acb1adc2cbfe\") " pod="kube-system/konnectivity-agent-dmc56" Jul 28 17:08:24.026588 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.026512 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c341008d-4e07-448d-9630-8d9a348088af-ovn-node-metrics-cert\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.042915 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.042888 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tg9b\" (UniqueName: \"kubernetes.io/projected/b71d4a48-0ead-41ac-be1b-ce910838865b-kube-api-access-9tg9b\") pod \"tuned-pf6xn\" (UID: \"b71d4a48-0ead-41ac-be1b-ce910838865b\") " pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:24.044331 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.044272 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v8gx\" (UniqueName: \"kubernetes.io/projected/c341008d-4e07-448d-9630-8d9a348088af-kube-api-access-9v8gx\") pod \"ovnkube-node-x6gbq\" (UID: \"c341008d-4e07-448d-9630-8d9a348088af\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.044331 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.044280 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nz59s\" (UniqueName: \"kubernetes.io/projected/2972842b-5159-4e7e-8330-da3425927ea0-kube-api-access-nz59s\") pod \"iptables-alerter-vmdd2\" (UID: \"2972842b-5159-4e7e-8330-da3425927ea0\") " pod="openshift-network-operator/iptables-alerter-vmdd2" Jul 28 17:08:24.098665 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.098591 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksszp" Jul 28 17:08:24.106865 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.106686 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-b9mxn" Jul 28 17:08:24.116031 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.116007 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jm8ng" Jul 28 17:08:24.120963 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.120912 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal" event={"ID":"29924a4ae605b9b2aa658a2296cc479f","Type":"ContainerStarted","Data":"f34963d1310f4af3e41eb5a10a0028f345b134c05e0e31e74c2bc2c7616ed23b"} Jul 28 17:08:24.122015 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.121984 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-11.ec2.internal" event={"ID":"a8cc528b9c87d3a6a58c64430cacff3d","Type":"ContainerStarted","Data":"c8cf06ab9de0018da8a22ca2448cb3c3a082bf9fd4bcb352f36620ff369f4e9e"} Jul 28 17:08:24.126275 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.126255 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zbmdl" Jul 28 17:08:24.131057 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.131040 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cmx6g" Jul 28 17:08:24.137779 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.137761 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dmc56" Jul 28 17:08:24.143451 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.143432 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" Jul 28 17:08:24.150996 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.150975 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:24.156911 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.156889 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vmdd2" Jul 28 17:08:24.425502 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.425461 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93b6b3b5-e860-4fee-9800-0f4534b72e67-metrics-certs\") pod \"network-metrics-daemon-5tdlw\" (UID: \"93b6b3b5-e860-4fee-9800-0f4534b72e67\") " pod="openshift-multus/network-metrics-daemon-5tdlw" Jul 28 17:08:24.425682 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:24.425637 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:08:24.425751 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:24.425701 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93b6b3b5-e860-4fee-9800-0f4534b72e67-metrics-certs podName:93b6b3b5-e860-4fee-9800-0f4534b72e67 nodeName:}" failed. No retries permitted until 2026-07-28 17:08:25.425687098 +0000 UTC m=+4.034618451 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/93b6b3b5-e860-4fee-9800-0f4534b72e67-metrics-certs") pod "network-metrics-daemon-5tdlw" (UID: "93b6b3b5-e860-4fee-9800-0f4534b72e67") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:08:24.507236 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:24.507203 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedd0a7b9_158c_4fdf_8069_5e49e0ce0772.slice/crio-24381d85db115e354a909aed91abef94038972b3b61810f4a9c62fa76e5f96d2 WatchSource:0}: Error finding container 24381d85db115e354a909aed91abef94038972b3b61810f4a9c62fa76e5f96d2: Status 404 returned error can't find the container with id 24381d85db115e354a909aed91abef94038972b3b61810f4a9c62fa76e5f96d2 Jul 28 17:08:24.526365 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.526339 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lzzz\" (UniqueName: \"kubernetes.io/projected/980a4184-ed46-42a1-99f3-86c49cfba43a-kube-api-access-7lzzz\") pod \"network-check-target-6dbsb\" (UID: \"980a4184-ed46-42a1-99f3-86c49cfba43a\") " pod="openshift-network-diagnostics/network-check-target-6dbsb" Jul 28 17:08:24.526484 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.526399 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9baa058e-a765-4c65-b5a6-ce0a7c022d97-original-pull-secret\") pod \"global-pull-secret-syncer-fbzsz\" (UID: \"9baa058e-a765-4c65-b5a6-ce0a7c022d97\") " pod="kube-system/global-pull-secret-syncer-fbzsz" Jul 28 17:08:24.526547 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:24.526483 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:08:24.526547 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:24.526506 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:08:24.526547 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:24.526527 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:08:24.526547 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:24.526540 2570 projected.go:196] Error preparing data for projected volume kube-api-access-7lzzz for pod openshift-network-diagnostics/network-check-target-6dbsb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:08:24.526731 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:24.526541 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9baa058e-a765-4c65-b5a6-ce0a7c022d97-original-pull-secret podName:9baa058e-a765-4c65-b5a6-ce0a7c022d97 nodeName:}" failed. No retries permitted until 2026-07-28 17:08:25.526520856 +0000 UTC m=+4.135452206 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9baa058e-a765-4c65-b5a6-ce0a7c022d97-original-pull-secret") pod "global-pull-secret-syncer-fbzsz" (UID: "9baa058e-a765-4c65-b5a6-ce0a7c022d97") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:08:24.526731 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:24.526609 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/980a4184-ed46-42a1-99f3-86c49cfba43a-kube-api-access-7lzzz podName:980a4184-ed46-42a1-99f3-86c49cfba43a nodeName:}" failed. No retries permitted until 2026-07-28 17:08:25.526595723 +0000 UTC m=+4.135527072 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-7lzzz" (UniqueName: "kubernetes.io/projected/980a4184-ed46-42a1-99f3-86c49cfba43a-kube-api-access-7lzzz") pod "network-check-target-6dbsb" (UID: "980a4184-ed46-42a1-99f3-86c49cfba43a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:08:24.863766 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.863684 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 17:03:24 +0000 UTC" deadline="2028-02-06 10:15:52.435957801 +0000 UTC" Jul 28 17:08:24.863766 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:24.863714 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13385h7m27.572247493s" Jul 28 17:08:25.113615 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:25.113580 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tdlw" Jul 28 17:08:25.113824 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:25.113714 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5tdlw" podUID="93b6b3b5-e860-4fee-9800-0f4534b72e67" Jul 28 17:08:25.125090 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:25.125051 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksszp" event={"ID":"edd0a7b9-158c-4fdf-8069-5e49e0ce0772","Type":"ContainerStarted","Data":"24381d85db115e354a909aed91abef94038972b3b61810f4a9c62fa76e5f96d2"} Jul 28 17:08:25.225796 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:25.225751 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75339f13_4483_4c0f_a502_885f0b22c8b7.slice/crio-4b3281666d8ef8011c146c613d1094645c64fb15b979d13db8ceae5ef96b8c53 WatchSource:0}: Error finding container 4b3281666d8ef8011c146c613d1094645c64fb15b979d13db8ceae5ef96b8c53: Status 404 returned error can't find the container with id 4b3281666d8ef8011c146c613d1094645c64fb15b979d13db8ceae5ef96b8c53 Jul 28 17:08:25.227614 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:25.227464 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2d09d1f_c076_4633_aa92_acb1adc2cbfe.slice/crio-329db2e6bb1338a39bfd66ee54c049dc97899795d169c813c1ed885d209fc654 WatchSource:0}: Error finding container 329db2e6bb1338a39bfd66ee54c049dc97899795d169c813c1ed885d209fc654: Status 404 returned error can't find the container with id 329db2e6bb1338a39bfd66ee54c049dc97899795d169c813c1ed885d209fc654 Jul 28 17:08:25.230496 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:25.230468 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56d1a325_976b_4060_b1a4_1a4b6be0bd18.slice/crio-0b17d9d5da227aa1fc7d4cea9e7bf432f752adf88244c145822d7fba7a5a5e1b WatchSource:0}: Error finding container 0b17d9d5da227aa1fc7d4cea9e7bf432f752adf88244c145822d7fba7a5a5e1b: Status 404 returned error can't find the container with id 0b17d9d5da227aa1fc7d4cea9e7bf432f752adf88244c145822d7fba7a5a5e1b Jul 28 17:08:25.231572 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:25.231547 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc341008d_4e07_448d_9630_8d9a348088af.slice/crio-3aac7e5f78975db0199ddd15cdf8af2e4bde328949d16a5fddab8c6dac76baf5 WatchSource:0}: Error finding container 3aac7e5f78975db0199ddd15cdf8af2e4bde328949d16a5fddab8c6dac76baf5: Status 404 returned error can't find the container with id 3aac7e5f78975db0199ddd15cdf8af2e4bde328949d16a5fddab8c6dac76baf5 Jul 28 17:08:25.234607 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:25.234072 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2972842b_5159_4e7e_8330_da3425927ea0.slice/crio-08b32cbfbb9cd3d968343d1d30649eea984f66149fc539ea4ec75cf66f72c548 WatchSource:0}: Error finding container 08b32cbfbb9cd3d968343d1d30649eea984f66149fc539ea4ec75cf66f72c548: Status 404 returned error can't find the container with id 08b32cbfbb9cd3d968343d1d30649eea984f66149fc539ea4ec75cf66f72c548 Jul 28 17:08:25.236786 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:25.236689 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36c66dd3_6c06_4883_8ccd_107c566a45d6.slice/crio-f571806047c63f3ecfb264aab358138569c6e520240a7159172b1ec254300788 WatchSource:0}: Error finding container f571806047c63f3ecfb264aab358138569c6e520240a7159172b1ec254300788: Status 404 returned error can't find the container with id f571806047c63f3ecfb264aab358138569c6e520240a7159172b1ec254300788 Jul 28 17:08:25.236934 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:25.236909 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd54c89e_8c35_404b_8804_ca3e45d384b4.slice/crio-21b6b89a3431dba4720afad37e75eef7835068486975cb78a45668952f898ad7 WatchSource:0}: Error finding container 21b6b89a3431dba4720afad37e75eef7835068486975cb78a45668952f898ad7: Status 404 returned error can't find the container with id 21b6b89a3431dba4720afad37e75eef7835068486975cb78a45668952f898ad7 Jul 28 17:08:25.237674 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:25.237650 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb71d4a48_0ead_41ac_be1b_ce910838865b.slice/crio-87492f1ce09f6d3ac0ffbbea44dc29a02e3144e118270e64dd4c7793dea2ce81 WatchSource:0}: Error finding container 87492f1ce09f6d3ac0ffbbea44dc29a02e3144e118270e64dd4c7793dea2ce81: Status 404 returned error can't find the container with id 87492f1ce09f6d3ac0ffbbea44dc29a02e3144e118270e64dd4c7793dea2ce81 Jul 28 17:08:25.433434 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:25.433170 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93b6b3b5-e860-4fee-9800-0f4534b72e67-metrics-certs\") pod \"network-metrics-daemon-5tdlw\" (UID: \"93b6b3b5-e860-4fee-9800-0f4534b72e67\") " pod="openshift-multus/network-metrics-daemon-5tdlw" Jul 28 17:08:25.433613 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:25.433341 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:08:25.433739 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:25.433636 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93b6b3b5-e860-4fee-9800-0f4534b72e67-metrics-certs podName:93b6b3b5-e860-4fee-9800-0f4534b72e67 nodeName:}" failed. No retries permitted until 2026-07-28 17:08:27.433609584 +0000 UTC m=+6.042540933 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/93b6b3b5-e860-4fee-9800-0f4534b72e67-metrics-certs") pod "network-metrics-daemon-5tdlw" (UID: "93b6b3b5-e860-4fee-9800-0f4534b72e67") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:08:25.534587 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:25.534552 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lzzz\" (UniqueName: \"kubernetes.io/projected/980a4184-ed46-42a1-99f3-86c49cfba43a-kube-api-access-7lzzz\") pod \"network-check-target-6dbsb\" (UID: \"980a4184-ed46-42a1-99f3-86c49cfba43a\") " pod="openshift-network-diagnostics/network-check-target-6dbsb" Jul 28 17:08:25.534711 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:25.534635 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9baa058e-a765-4c65-b5a6-ce0a7c022d97-original-pull-secret\") pod \"global-pull-secret-syncer-fbzsz\" (UID: \"9baa058e-a765-4c65-b5a6-ce0a7c022d97\") " pod="kube-system/global-pull-secret-syncer-fbzsz" Jul 28 17:08:25.534767 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:25.534730 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:08:25.534800 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:25.534781 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9baa058e-a765-4c65-b5a6-ce0a7c022d97-original-pull-secret podName:9baa058e-a765-4c65-b5a6-ce0a7c022d97 nodeName:}" failed. No retries permitted until 2026-07-28 17:08:27.534767574 +0000 UTC m=+6.143698924 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9baa058e-a765-4c65-b5a6-ce0a7c022d97-original-pull-secret") pod "global-pull-secret-syncer-fbzsz" (UID: "9baa058e-a765-4c65-b5a6-ce0a7c022d97") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:08:25.534852 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:25.534733 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:08:25.534852 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:25.534835 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:08:25.534852 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:25.534849 2570 projected.go:196] Error preparing data for projected volume kube-api-access-7lzzz for pod openshift-network-diagnostics/network-check-target-6dbsb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:08:25.534947 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:25.534894 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/980a4184-ed46-42a1-99f3-86c49cfba43a-kube-api-access-7lzzz podName:980a4184-ed46-42a1-99f3-86c49cfba43a nodeName:}" failed. No retries permitted until 2026-07-28 17:08:27.53488104 +0000 UTC m=+6.143812411 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-7lzzz" (UniqueName: "kubernetes.io/projected/980a4184-ed46-42a1-99f3-86c49cfba43a-kube-api-access-7lzzz") pod "network-check-target-6dbsb" (UID: "980a4184-ed46-42a1-99f3-86c49cfba43a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:08:25.864947 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:25.864793 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 17:03:24 +0000 UTC" deadline="2028-04-14 08:53:28.263453486 +0000 UTC" Jul 28 17:08:25.864947 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:25.864863 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15015h45m2.39859499s" Jul 28 17:08:26.115271 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:26.113942 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbzsz" Jul 28 17:08:26.115271 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:26.114074 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fbzsz" podUID="9baa058e-a765-4c65-b5a6-ce0a7c022d97" Jul 28 17:08:26.115271 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:26.114509 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6dbsb" Jul 28 17:08:26.115271 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:26.114603 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6dbsb" podUID="980a4184-ed46-42a1-99f3-86c49cfba43a" Jul 28 17:08:26.136841 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:26.136801 2570 generic.go:356] "Generic (PLEG): container finished" podID="29924a4ae605b9b2aa658a2296cc479f" containerID="e61385d387255a01541956da21876916a30722e62b467981412e5400ee9cdf4f" exitCode=0 Jul 28 17:08:26.137002 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:26.136900 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal" event={"ID":"29924a4ae605b9b2aa658a2296cc479f","Type":"ContainerDied","Data":"e61385d387255a01541956da21876916a30722e62b467981412e5400ee9cdf4f"} Jul 28 17:08:26.154114 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:26.154085 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-11.ec2.internal" event={"ID":"a8cc528b9c87d3a6a58c64430cacff3d","Type":"ContainerStarted","Data":"a0bdb9666d0c28d6302b7126d0ed567377b23a4d2686efd8e377ed4e8348d801"} Jul 28 17:08:26.158703 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:26.158652 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" event={"ID":"b71d4a48-0ead-41ac-be1b-ce910838865b","Type":"ContainerStarted","Data":"87492f1ce09f6d3ac0ffbbea44dc29a02e3144e118270e64dd4c7793dea2ce81"} Jul 28 17:08:26.167151 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:26.167120 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" event={"ID":"c341008d-4e07-448d-9630-8d9a348088af","Type":"ContainerStarted","Data":"3aac7e5f78975db0199ddd15cdf8af2e4bde328949d16a5fddab8c6dac76baf5"} Jul 28 17:08:26.171823 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:26.171796 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vmdd2" event={"ID":"2972842b-5159-4e7e-8330-da3425927ea0","Type":"ContainerStarted","Data":"08b32cbfbb9cd3d968343d1d30649eea984f66149fc539ea4ec75cf66f72c548"} Jul 28 17:08:26.176241 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:26.176217 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cmx6g" event={"ID":"56d1a325-976b-4060-b1a4-1a4b6be0bd18","Type":"ContainerStarted","Data":"0b17d9d5da227aa1fc7d4cea9e7bf432f752adf88244c145822d7fba7a5a5e1b"} Jul 28 17:08:26.190346 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:26.190322 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dmc56" event={"ID":"e2d09d1f-c076-4633-aa92-acb1adc2cbfe","Type":"ContainerStarted","Data":"329db2e6bb1338a39bfd66ee54c049dc97899795d169c813c1ed885d209fc654"} Jul 28 17:08:26.194431 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:26.194360 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zbmdl" event={"ID":"dd54c89e-8c35-404b-8804-ca3e45d384b4","Type":"ContainerStarted","Data":"21b6b89a3431dba4720afad37e75eef7835068486975cb78a45668952f898ad7"} Jul 28 17:08:26.205934 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:26.205834 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b9mxn" event={"ID":"36c66dd3-6c06-4883-8ccd-107c566a45d6","Type":"ContainerStarted","Data":"f571806047c63f3ecfb264aab358138569c6e520240a7159172b1ec254300788"} Jul 28 17:08:26.230397 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:26.230327 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jm8ng" event={"ID":"75339f13-4483-4c0f-a502-885f0b22c8b7","Type":"ContainerStarted","Data":"4b3281666d8ef8011c146c613d1094645c64fb15b979d13db8ceae5ef96b8c53"} Jul 28 17:08:27.114588 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:27.114492 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tdlw" Jul 28 17:08:27.115111 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:27.114647 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5tdlw" podUID="93b6b3b5-e860-4fee-9800-0f4534b72e67" Jul 28 17:08:27.269333 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:27.269255 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal" event={"ID":"29924a4ae605b9b2aa658a2296cc479f","Type":"ContainerStarted","Data":"51121357b8b37d01abb85209e48ab743717364b3e620e985ea53bbfbb86e869d"} Jul 28 17:08:27.290930 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:27.290868 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-11.ec2.internal" podStartSLOduration=4.290848422 podStartE2EDuration="4.290848422s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:08:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:08:26.188744479 +0000 UTC m=+4.797675853" watchObservedRunningTime="2026-07-28 17:08:27.290848422 +0000 UTC m=+5.899779797" Jul 28 17:08:27.448442 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:27.448404 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93b6b3b5-e860-4fee-9800-0f4534b72e67-metrics-certs\") pod \"network-metrics-daemon-5tdlw\" (UID: \"93b6b3b5-e860-4fee-9800-0f4534b72e67\") " pod="openshift-multus/network-metrics-daemon-5tdlw" Jul 28 17:08:27.448635 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:27.448586 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:08:27.448695 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:27.448651 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93b6b3b5-e860-4fee-9800-0f4534b72e67-metrics-certs podName:93b6b3b5-e860-4fee-9800-0f4534b72e67 nodeName:}" failed. No retries permitted until 2026-07-28 17:08:31.448632056 +0000 UTC m=+10.057563420 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/93b6b3b5-e860-4fee-9800-0f4534b72e67-metrics-certs") pod "network-metrics-daemon-5tdlw" (UID: "93b6b3b5-e860-4fee-9800-0f4534b72e67") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:08:27.549731 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:27.549684 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9baa058e-a765-4c65-b5a6-ce0a7c022d97-original-pull-secret\") pod \"global-pull-secret-syncer-fbzsz\" (UID: \"9baa058e-a765-4c65-b5a6-ce0a7c022d97\") " pod="kube-system/global-pull-secret-syncer-fbzsz" Jul 28 17:08:27.549923 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:27.549754 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lzzz\" (UniqueName: \"kubernetes.io/projected/980a4184-ed46-42a1-99f3-86c49cfba43a-kube-api-access-7lzzz\") pod \"network-check-target-6dbsb\" (UID: \"980a4184-ed46-42a1-99f3-86c49cfba43a\") " pod="openshift-network-diagnostics/network-check-target-6dbsb" Jul 28 17:08:27.549995 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:27.549933 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:08:27.549995 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:27.549952 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:08:27.549995 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:27.549968 2570 projected.go:196] Error preparing data for projected volume kube-api-access-7lzzz for pod openshift-network-diagnostics/network-check-target-6dbsb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:08:27.550164 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:27.550027 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/980a4184-ed46-42a1-99f3-86c49cfba43a-kube-api-access-7lzzz podName:980a4184-ed46-42a1-99f3-86c49cfba43a nodeName:}" failed. No retries permitted until 2026-07-28 17:08:31.550007618 +0000 UTC m=+10.158938971 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-7lzzz" (UniqueName: "kubernetes.io/projected/980a4184-ed46-42a1-99f3-86c49cfba43a-kube-api-access-7lzzz") pod "network-check-target-6dbsb" (UID: "980a4184-ed46-42a1-99f3-86c49cfba43a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:08:27.550530 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:27.550507 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:08:27.550636 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:27.550570 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9baa058e-a765-4c65-b5a6-ce0a7c022d97-original-pull-secret podName:9baa058e-a765-4c65-b5a6-ce0a7c022d97 nodeName:}" failed. No retries permitted until 2026-07-28 17:08:31.550553189 +0000 UTC m=+10.159484543 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9baa058e-a765-4c65-b5a6-ce0a7c022d97-original-pull-secret") pod "global-pull-secret-syncer-fbzsz" (UID: "9baa058e-a765-4c65-b5a6-ce0a7c022d97") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:08:28.113908 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:28.113867 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbzsz" Jul 28 17:08:28.114162 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:28.114019 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fbzsz" podUID="9baa058e-a765-4c65-b5a6-ce0a7c022d97" Jul 28 17:08:28.114619 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:28.114498 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6dbsb" Jul 28 17:08:28.114619 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:28.114608 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6dbsb" podUID="980a4184-ed46-42a1-99f3-86c49cfba43a" Jul 28 17:08:29.113827 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:29.113300 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tdlw" Jul 28 17:08:29.113827 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:29.113448 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5tdlw" podUID="93b6b3b5-e860-4fee-9800-0f4534b72e67" Jul 28 17:08:30.118631 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:30.118596 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6dbsb" Jul 28 17:08:30.119132 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:30.118731 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6dbsb" podUID="980a4184-ed46-42a1-99f3-86c49cfba43a" Jul 28 17:08:30.119132 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:30.118932 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbzsz" Jul 28 17:08:30.119132 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:30.119026 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fbzsz" podUID="9baa058e-a765-4c65-b5a6-ce0a7c022d97" Jul 28 17:08:31.114360 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:31.114327 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tdlw" Jul 28 17:08:31.114539 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:31.114471 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5tdlw" podUID="93b6b3b5-e860-4fee-9800-0f4534b72e67" Jul 28 17:08:31.486304 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:31.486264 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93b6b3b5-e860-4fee-9800-0f4534b72e67-metrics-certs\") pod \"network-metrics-daemon-5tdlw\" (UID: \"93b6b3b5-e860-4fee-9800-0f4534b72e67\") " pod="openshift-multus/network-metrics-daemon-5tdlw" Jul 28 17:08:31.486892 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:31.486473 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:08:31.486892 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:31.486533 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93b6b3b5-e860-4fee-9800-0f4534b72e67-metrics-certs podName:93b6b3b5-e860-4fee-9800-0f4534b72e67 nodeName:}" failed. No retries permitted until 2026-07-28 17:08:39.48651337 +0000 UTC m=+18.095444734 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/93b6b3b5-e860-4fee-9800-0f4534b72e67-metrics-certs") pod "network-metrics-daemon-5tdlw" (UID: "93b6b3b5-e860-4fee-9800-0f4534b72e67") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:08:31.587439 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:31.587396 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lzzz\" (UniqueName: \"kubernetes.io/projected/980a4184-ed46-42a1-99f3-86c49cfba43a-kube-api-access-7lzzz\") pod \"network-check-target-6dbsb\" (UID: \"980a4184-ed46-42a1-99f3-86c49cfba43a\") " pod="openshift-network-diagnostics/network-check-target-6dbsb" Jul 28 17:08:31.587628 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:31.587509 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9baa058e-a765-4c65-b5a6-ce0a7c022d97-original-pull-secret\") pod \"global-pull-secret-syncer-fbzsz\" (UID: \"9baa058e-a765-4c65-b5a6-ce0a7c022d97\") " pod="kube-system/global-pull-secret-syncer-fbzsz" Jul 28 17:08:31.587711 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:31.587648 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:08:31.587775 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:31.587714 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9baa058e-a765-4c65-b5a6-ce0a7c022d97-original-pull-secret podName:9baa058e-a765-4c65-b5a6-ce0a7c022d97 nodeName:}" failed. No retries permitted until 2026-07-28 17:08:39.587692627 +0000 UTC m=+18.196623978 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9baa058e-a765-4c65-b5a6-ce0a7c022d97-original-pull-secret") pod "global-pull-secret-syncer-fbzsz" (UID: "9baa058e-a765-4c65-b5a6-ce0a7c022d97") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:08:31.588263 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:31.588152 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:08:31.588263 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:31.588180 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:08:31.588263 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:31.588211 2570 projected.go:196] Error preparing data for projected volume kube-api-access-7lzzz for pod openshift-network-diagnostics/network-check-target-6dbsb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:08:31.588263 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:31.588261 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/980a4184-ed46-42a1-99f3-86c49cfba43a-kube-api-access-7lzzz podName:980a4184-ed46-42a1-99f3-86c49cfba43a nodeName:}" failed. No retries permitted until 2026-07-28 17:08:39.588245482 +0000 UTC m=+18.197176849 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-7lzzz" (UniqueName: "kubernetes.io/projected/980a4184-ed46-42a1-99f3-86c49cfba43a-kube-api-access-7lzzz") pod "network-check-target-6dbsb" (UID: "980a4184-ed46-42a1-99f3-86c49cfba43a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:08:32.117685 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:32.115538 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6dbsb" Jul 28 17:08:32.117685 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:32.115731 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6dbsb" podUID="980a4184-ed46-42a1-99f3-86c49cfba43a" Jul 28 17:08:32.117685 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:32.115837 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbzsz" Jul 28 17:08:32.117685 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:32.115963 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fbzsz" podUID="9baa058e-a765-4c65-b5a6-ce0a7c022d97" Jul 28 17:08:33.114126 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:33.114077 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tdlw" Jul 28 17:08:33.114590 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:33.114238 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5tdlw" podUID="93b6b3b5-e860-4fee-9800-0f4534b72e67" Jul 28 17:08:34.113958 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:34.113830 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbzsz" Jul 28 17:08:34.114150 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:34.113963 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fbzsz" podUID="9baa058e-a765-4c65-b5a6-ce0a7c022d97" Jul 28 17:08:34.114150 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:34.113830 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6dbsb" Jul 28 17:08:34.114150 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:34.114055 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6dbsb" podUID="980a4184-ed46-42a1-99f3-86c49cfba43a" Jul 28 17:08:35.113448 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:35.113414 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tdlw" Jul 28 17:08:35.113609 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:35.113545 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5tdlw" podUID="93b6b3b5-e860-4fee-9800-0f4534b72e67" Jul 28 17:08:36.113706 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:36.113673 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6dbsb" Jul 28 17:08:36.114194 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:36.113806 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6dbsb" podUID="980a4184-ed46-42a1-99f3-86c49cfba43a" Jul 28 17:08:36.114194 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:36.113846 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbzsz" Jul 28 17:08:36.114194 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:36.113931 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fbzsz" podUID="9baa058e-a765-4c65-b5a6-ce0a7c022d97" Jul 28 17:08:37.113356 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:37.113324 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tdlw" Jul 28 17:08:37.113534 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:37.113445 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5tdlw" podUID="93b6b3b5-e860-4fee-9800-0f4534b72e67" Jul 28 17:08:38.113501 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:38.113466 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6dbsb" Jul 28 17:08:38.113917 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:38.113576 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6dbsb" podUID="980a4184-ed46-42a1-99f3-86c49cfba43a" Jul 28 17:08:38.113917 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:38.113608 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbzsz" Jul 28 17:08:38.113917 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:38.113678 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fbzsz" podUID="9baa058e-a765-4c65-b5a6-ce0a7c022d97" Jul 28 17:08:39.113239 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:39.113206 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tdlw" Jul 28 17:08:39.113529 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:39.113343 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5tdlw" podUID="93b6b3b5-e860-4fee-9800-0f4534b72e67" Jul 28 17:08:39.551186 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:39.551142 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93b6b3b5-e860-4fee-9800-0f4534b72e67-metrics-certs\") pod \"network-metrics-daemon-5tdlw\" (UID: \"93b6b3b5-e860-4fee-9800-0f4534b72e67\") " pod="openshift-multus/network-metrics-daemon-5tdlw" Jul 28 17:08:39.551389 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:39.551304 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:08:39.551389 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:39.551388 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93b6b3b5-e860-4fee-9800-0f4534b72e67-metrics-certs podName:93b6b3b5-e860-4fee-9800-0f4534b72e67 nodeName:}" failed. No retries permitted until 2026-07-28 17:08:55.551365328 +0000 UTC m=+34.160296677 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/93b6b3b5-e860-4fee-9800-0f4534b72e67-metrics-certs") pod "network-metrics-daemon-5tdlw" (UID: "93b6b3b5-e860-4fee-9800-0f4534b72e67") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:08:39.652399 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:39.652369 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lzzz\" (UniqueName: \"kubernetes.io/projected/980a4184-ed46-42a1-99f3-86c49cfba43a-kube-api-access-7lzzz\") pod \"network-check-target-6dbsb\" (UID: \"980a4184-ed46-42a1-99f3-86c49cfba43a\") " pod="openshift-network-diagnostics/network-check-target-6dbsb" Jul 28 17:08:39.652562 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:39.652445 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9baa058e-a765-4c65-b5a6-ce0a7c022d97-original-pull-secret\") pod \"global-pull-secret-syncer-fbzsz\" (UID: \"9baa058e-a765-4c65-b5a6-ce0a7c022d97\") " pod="kube-system/global-pull-secret-syncer-fbzsz" Jul 28 17:08:39.652562 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:39.652551 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:08:39.652646 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:39.652558 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:08:39.652646 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:39.652583 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:08:39.652646 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:39.652592 2570 projected.go:196] Error preparing data for projected volume kube-api-access-7lzzz for pod openshift-network-diagnostics/network-check-target-6dbsb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:08:39.652646 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:39.652603 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9baa058e-a765-4c65-b5a6-ce0a7c022d97-original-pull-secret podName:9baa058e-a765-4c65-b5a6-ce0a7c022d97 nodeName:}" failed. No retries permitted until 2026-07-28 17:08:55.652590511 +0000 UTC m=+34.261521863 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9baa058e-a765-4c65-b5a6-ce0a7c022d97-original-pull-secret") pod "global-pull-secret-syncer-fbzsz" (UID: "9baa058e-a765-4c65-b5a6-ce0a7c022d97") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:08:39.652646 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:39.652634 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/980a4184-ed46-42a1-99f3-86c49cfba43a-kube-api-access-7lzzz podName:980a4184-ed46-42a1-99f3-86c49cfba43a nodeName:}" failed. No retries permitted until 2026-07-28 17:08:55.652618544 +0000 UTC m=+34.261549892 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-7lzzz" (UniqueName: "kubernetes.io/projected/980a4184-ed46-42a1-99f3-86c49cfba43a-kube-api-access-7lzzz") pod "network-check-target-6dbsb" (UID: "980a4184-ed46-42a1-99f3-86c49cfba43a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:08:40.113964 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:40.113934 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6dbsb" Jul 28 17:08:40.113964 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:40.113955 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbzsz" Jul 28 17:08:40.114487 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:40.114028 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6dbsb" podUID="980a4184-ed46-42a1-99f3-86c49cfba43a" Jul 28 17:08:40.114487 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:40.114159 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fbzsz" podUID="9baa058e-a765-4c65-b5a6-ce0a7c022d97" Jul 28 17:08:41.114357 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:41.114309 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tdlw" Jul 28 17:08:41.114801 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:41.114450 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5tdlw" podUID="93b6b3b5-e860-4fee-9800-0f4534b72e67" Jul 28 17:08:42.114609 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:42.114575 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbzsz" Jul 28 17:08:42.115047 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:42.114646 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6dbsb" Jul 28 17:08:42.115047 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:42.114666 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fbzsz" podUID="9baa058e-a765-4c65-b5a6-ce0a7c022d97" Jul 28 17:08:42.115047 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:42.114737 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6dbsb" podUID="980a4184-ed46-42a1-99f3-86c49cfba43a" Jul 28 17:08:43.113677 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:43.113640 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tdlw" Jul 28 17:08:43.113858 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:43.113791 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5tdlw" podUID="93b6b3b5-e860-4fee-9800-0f4534b72e67" Jul 28 17:08:44.114685 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:44.114640 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbzsz" Jul 28 17:08:44.114685 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:44.114677 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6dbsb" Jul 28 17:08:44.115492 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:44.115292 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6dbsb" podUID="980a4184-ed46-42a1-99f3-86c49cfba43a" Jul 28 17:08:44.115590 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:44.114780 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fbzsz" podUID="9baa058e-a765-4c65-b5a6-ce0a7c022d97" Jul 28 17:08:44.313051 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:44.312844 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksszp" event={"ID":"edd0a7b9-158c-4fdf-8069-5e49e0ce0772","Type":"ContainerStarted","Data":"8d1a811893cd149c76557dddc7f44d02f0aa30ec552adf6c27238b0d0592d95c"} Jul 28 17:08:44.315715 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:44.315222 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" event={"ID":"b71d4a48-0ead-41ac-be1b-ce910838865b","Type":"ContainerStarted","Data":"a802b879cc55356c06739cf1b02f3b84c8e200f6789038998cf920361ba71cca"} Jul 28 17:08:44.317373 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:44.317297 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" event={"ID":"c341008d-4e07-448d-9630-8d9a348088af","Type":"ContainerStarted","Data":"aa24157a15670c8e3da2cf68dba6ddac193a668210cf5989459304943e3ae580"} Jul 28 17:08:44.318795 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:44.318757 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cmx6g" event={"ID":"56d1a325-976b-4060-b1a4-1a4b6be0bd18","Type":"ContainerStarted","Data":"67beea9832d307d3bf61ca364f512f1287b4ee41d2b2b01d129abdeaaae8ef15"} Jul 28 17:08:44.320259 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:44.320235 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dmc56" event={"ID":"e2d09d1f-c076-4633-aa92-acb1adc2cbfe","Type":"ContainerStarted","Data":"f8fc2e863947a26ae65ff8fdfbe6c45dc458411254f10a753397b7e75a35b279"} Jul 28 17:08:44.322571 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:44.322261 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zbmdl" event={"ID":"dd54c89e-8c35-404b-8804-ca3e45d384b4","Type":"ContainerStarted","Data":"f9cc92223e64792bd7cb681b1265df7d6ed12b304de559cbd366f7a3a15f8396"} Jul 28 17:08:44.324415 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:44.324368 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b9mxn" event={"ID":"36c66dd3-6c06-4883-8ccd-107c566a45d6","Type":"ContainerStarted","Data":"ae3acf52ca3ce4810d1285140e98a4e46cce8e39cd70673cdfae7486e1aa9394"} Jul 28 17:08:44.340292 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:44.340248 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-pf6xn" podStartSLOduration=2.481165936 podStartE2EDuration="21.340234307s" totalImagesPullingTime="18.859068371s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:08:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:08:44.339892938 +0000 UTC m=+22.948824308" watchObservedRunningTime="2026-07-28 17:08:44.340234307 +0000 UTC m=+22.949165655" Jul 28 17:08:44.340399 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:44.340381 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal" podStartSLOduration=21.340376453 podStartE2EDuration="21.340376453s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:08:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:08:27.291242474 +0000 UTC m=+5.900173849" watchObservedRunningTime="2026-07-28 17:08:44.340376453 +0000 UTC m=+22.949307823" Jul 28 17:08:44.405927 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:44.405646 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-zbmdl" podStartSLOduration=3.548736901 podStartE2EDuration="22.405604002s" totalImagesPullingTime="18.856867101s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:08:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:08:44.404302272 +0000 UTC m=+23.013233662" watchObservedRunningTime="2026-07-28 17:08:44.405604002 +0000 UTC m=+23.014535372" Jul 28 17:08:44.426292 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:44.426241 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-jm8ng" podStartSLOduration=3.472260178 podStartE2EDuration="22.426225336s" totalImagesPullingTime="18.953965158s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:08:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:08:44.425881892 +0000 UTC m=+23.034813261" watchObservedRunningTime="2026-07-28 17:08:44.426225336 +0000 UTC m=+23.035156706" Jul 28 17:08:44.466308 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:44.466261 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-dmc56" podStartSLOduration=7.574817452 podStartE2EDuration="22.466245426s" totalImagesPullingTime="14.891427974s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:08:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:08:44.465431393 +0000 UTC m=+23.074362762" watchObservedRunningTime="2026-07-28 17:08:44.466245426 +0000 UTC m=+23.075176796" Jul 28 17:08:44.466560 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:44.466538 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-cmx6g" podStartSLOduration=3.580734166 podStartE2EDuration="22.466532468s" totalImagesPullingTime="18.885798302s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:08:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:08:44.44641508 +0000 UTC m=+23.055346451" watchObservedRunningTime="2026-07-28 17:08:44.466532468 +0000 UTC m=+23.075463838" Jul 28 17:08:45.113284 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:45.113223 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tdlw" Jul 28 17:08:45.113409 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:45.113328 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5tdlw" podUID="93b6b3b5-e860-4fee-9800-0f4534b72e67" Jul 28 17:08:45.331149 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:45.331098 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" event={"ID":"c341008d-4e07-448d-9630-8d9a348088af","Type":"ContainerStarted","Data":"42ab2a90b0f7cac16709c77e3c148d2f006499000977a6b3137c761714016416"} Jul 28 17:08:45.331939 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:45.331213 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" event={"ID":"c341008d-4e07-448d-9630-8d9a348088af","Type":"ContainerStarted","Data":"061bbdbb0ccb78cb3e3c937e1c5f155b4c20feb932d1d44ad5fc50c91effcda8"} Jul 28 17:08:45.331939 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:45.331253 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" event={"ID":"c341008d-4e07-448d-9630-8d9a348088af","Type":"ContainerStarted","Data":"0b1e0edae04270c77f815955ff8624e47fda6072cf4d78a91cd6ae3cd1c7662c"} Jul 28 17:08:45.331939 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:45.331282 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" event={"ID":"c341008d-4e07-448d-9630-8d9a348088af","Type":"ContainerStarted","Data":"10a6df2e469b4897d1c84680c636526da17d1305b132cbb951362a1f8538d4d1"} Jul 28 17:08:45.331939 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:45.331325 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" event={"ID":"c341008d-4e07-448d-9630-8d9a348088af","Type":"ContainerStarted","Data":"38365b89186aea1d440ed424ae7b73a4f0a457cdd15a62c5ff86e897808a4a4f"} Jul 28 17:08:45.332464 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:45.332444 2570 generic.go:356] "Generic (PLEG): container finished" podID="36c66dd3-6c06-4883-8ccd-107c566a45d6" containerID="ae3acf52ca3ce4810d1285140e98a4e46cce8e39cd70673cdfae7486e1aa9394" exitCode=0 Jul 28 17:08:45.332521 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:45.332508 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b9mxn" event={"ID":"36c66dd3-6c06-4883-8ccd-107c566a45d6","Type":"ContainerDied","Data":"ae3acf52ca3ce4810d1285140e98a4e46cce8e39cd70673cdfae7486e1aa9394"} Jul 28 17:08:45.334000 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:45.333979 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jm8ng" event={"ID":"75339f13-4483-4c0f-a502-885f0b22c8b7","Type":"ContainerStarted","Data":"edfa7799fbe4a05ed2f547facb61d33d99a1d6e084d56c873719b29ffcf40886"} Jul 28 17:08:45.494335 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:45.494313 2570 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 28 17:08:45.886635 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:45.886541 2570 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T17:08:45.49433066Z","UUID":"1d4edf56-db7e-4e61-8537-689ada0e606a","Handler":null,"Name":"","Endpoint":""} Jul 28 17:08:45.887885 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:45.887865 2570 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 28 17:08:45.887986 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:45.887900 2570 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 28 17:08:46.117602 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:46.117574 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6dbsb" Jul 28 17:08:46.117751 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:46.117672 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6dbsb" podUID="980a4184-ed46-42a1-99f3-86c49cfba43a" Jul 28 17:08:46.117830 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:46.117773 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbzsz" Jul 28 17:08:46.117905 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:46.117885 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fbzsz" podUID="9baa058e-a765-4c65-b5a6-ce0a7c022d97" Jul 28 17:08:46.337792 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:46.337702 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksszp" event={"ID":"edd0a7b9-158c-4fdf-8069-5e49e0ce0772","Type":"ContainerStarted","Data":"05842cb39d223ef5b4104edcd42b3de1166b1151712cb2dcf8824d5390709fe4"} Jul 28 17:08:46.339241 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:46.339213 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vmdd2" event={"ID":"2972842b-5159-4e7e-8330-da3425927ea0","Type":"ContainerStarted","Data":"4076458ee38d03efd67aab3dc6d411f0971a14c0f38e71ee8ec74ab21df7f72e"} Jul 28 17:08:46.851707 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:46.851677 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-dmc56" Jul 28 17:08:46.852266 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:46.852244 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-dmc56" Jul 28 17:08:46.879316 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:46.879280 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-vmdd2" podStartSLOduration=5.021797622 podStartE2EDuration="23.879266719s" totalImagesPullingTime="18.857469097s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:08:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:08:46.362362408 +0000 UTC m=+24.971293778" watchObservedRunningTime="2026-07-28 17:08:46.879266719 +0000 UTC m=+25.488198089" Jul 28 17:08:47.114030 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:47.114002 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tdlw" Jul 28 17:08:47.114210 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:47.114130 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5tdlw" podUID="93b6b3b5-e860-4fee-9800-0f4534b72e67" Jul 28 17:08:47.343903 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:47.343823 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksszp" event={"ID":"edd0a7b9-158c-4fdf-8069-5e49e0ce0772","Type":"ContainerStarted","Data":"4025731d096ca7e874a0ff74517410c5a93c8fd7c9885817c990ef70e6397063"} Jul 28 17:08:47.347889 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:47.347860 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" event={"ID":"c341008d-4e07-448d-9630-8d9a348088af","Type":"ContainerStarted","Data":"4f295b3ccc582142d66c9ccccd68526821f3f7ceeab58a7f436dafbace543d8f"} Jul 28 17:08:47.348281 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:47.348237 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-dmc56" Jul 28 17:08:47.348730 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:47.348699 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-dmc56" Jul 28 17:08:47.371629 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:47.371582 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksszp" podStartSLOduration=3.618629051 podStartE2EDuration="25.371566778s" totalImagesPullingTime="21.752937727s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:08:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:08:47.370827817 +0000 UTC m=+25.979759187" watchObservedRunningTime="2026-07-28 17:08:47.371566778 +0000 UTC m=+25.980498158" Jul 28 17:08:48.117350 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:48.117174 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6dbsb" Jul 28 17:08:48.117520 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:48.117195 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbzsz" Jul 28 17:08:48.117520 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:48.117446 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6dbsb" podUID="980a4184-ed46-42a1-99f3-86c49cfba43a" Jul 28 17:08:48.117631 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:48.117528 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fbzsz" podUID="9baa058e-a765-4c65-b5a6-ce0a7c022d97" Jul 28 17:08:49.113828 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:49.113792 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tdlw" Jul 28 17:08:49.114340 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:49.113928 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5tdlw" podUID="93b6b3b5-e860-4fee-9800-0f4534b72e67" Jul 28 17:08:50.116944 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:50.116712 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6dbsb" Jul 28 17:08:50.117716 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:50.116769 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbzsz" Jul 28 17:08:50.117716 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:50.116984 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6dbsb" podUID="980a4184-ed46-42a1-99f3-86c49cfba43a" Jul 28 17:08:50.117716 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:50.117024 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fbzsz" podUID="9baa058e-a765-4c65-b5a6-ce0a7c022d97" Jul 28 17:08:50.358249 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:50.358219 2570 generic.go:356] "Generic (PLEG): container finished" podID="36c66dd3-6c06-4883-8ccd-107c566a45d6" containerID="79fe8f77b6243cabfe748556750255af12716418a32cc5cfc108fa573ff2e81a" exitCode=0 Jul 28 17:08:50.358448 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:50.358314 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b9mxn" event={"ID":"36c66dd3-6c06-4883-8ccd-107c566a45d6","Type":"ContainerDied","Data":"79fe8f77b6243cabfe748556750255af12716418a32cc5cfc108fa573ff2e81a"} Jul 28 17:08:50.362183 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:50.362160 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" event={"ID":"c341008d-4e07-448d-9630-8d9a348088af","Type":"ContainerStarted","Data":"40a826875284f51f4f78503f0e177590276764d21d66b8aeb703c0c6c2740ab8"} Jul 28 17:08:50.362430 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:50.362409 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:50.362541 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:50.362498 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:50.362623 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:50.362557 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:50.377470 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:50.377451 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:50.377583 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:50.377543 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:08:51.113895 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:51.113862 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tdlw" Jul 28 17:08:51.114097 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:51.113970 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5tdlw" podUID="93b6b3b5-e860-4fee-9800-0f4534b72e67" Jul 28 17:08:51.367396 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:51.367313 2570 generic.go:356] "Generic (PLEG): container finished" podID="36c66dd3-6c06-4883-8ccd-107c566a45d6" containerID="1aa77cb25dd4f75671de6b40053f2e04ce4092461804daa10316c6db2c641b67" exitCode=0 Jul 28 17:08:51.367809 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:51.367398 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b9mxn" event={"ID":"36c66dd3-6c06-4883-8ccd-107c566a45d6","Type":"ContainerDied","Data":"1aa77cb25dd4f75671de6b40053f2e04ce4092461804daa10316c6db2c641b67"} Jul 28 17:08:51.397971 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:51.397914 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" podStartSLOduration=9.450748221 podStartE2EDuration="28.397900645s" totalImagesPullingTime="18.947152424s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:08:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:08:50.423728744 +0000 UTC m=+29.032660113" watchObservedRunningTime="2026-07-28 17:08:51.397900645 +0000 UTC m=+30.006832038" Jul 28 17:08:51.615535 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:51.615496 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6dbsb"] Jul 28 17:08:51.615709 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:51.615634 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6dbsb" Jul 28 17:08:51.615752 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:51.615715 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6dbsb" podUID="980a4184-ed46-42a1-99f3-86c49cfba43a" Jul 28 17:08:51.617818 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:51.617762 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fbzsz"] Jul 28 17:08:51.617934 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:51.617864 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbzsz" Jul 28 17:08:51.617995 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:51.617958 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fbzsz" podUID="9baa058e-a765-4c65-b5a6-ce0a7c022d97" Jul 28 17:08:51.620543 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:51.620520 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5tdlw"] Jul 28 17:08:51.620648 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:51.620608 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tdlw" Jul 28 17:08:51.620709 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:51.620693 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5tdlw" podUID="93b6b3b5-e860-4fee-9800-0f4534b72e67" Jul 28 17:08:52.372434 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:52.372384 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b9mxn" event={"ID":"36c66dd3-6c06-4883-8ccd-107c566a45d6","Type":"ContainerStarted","Data":"d74405c79cd7631f8f26e3673b618b643f4532618091b00f5cd63d3f09aeb2eb"} Jul 28 17:08:53.114150 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:53.114124 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbzsz" Jul 28 17:08:53.114263 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:53.114167 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6dbsb" Jul 28 17:08:53.114263 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:53.114226 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tdlw" Jul 28 17:08:53.114263 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:53.114243 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fbzsz" podUID="9baa058e-a765-4c65-b5a6-ce0a7c022d97" Jul 28 17:08:53.114423 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:53.114304 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5tdlw" podUID="93b6b3b5-e860-4fee-9800-0f4534b72e67" Jul 28 17:08:53.114423 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:53.114383 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6dbsb" podUID="980a4184-ed46-42a1-99f3-86c49cfba43a" Jul 28 17:08:53.377720 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:53.377470 2570 generic.go:356] "Generic (PLEG): container finished" podID="36c66dd3-6c06-4883-8ccd-107c566a45d6" containerID="d74405c79cd7631f8f26e3673b618b643f4532618091b00f5cd63d3f09aeb2eb" exitCode=0 Jul 28 17:08:53.377720 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:53.377533 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b9mxn" event={"ID":"36c66dd3-6c06-4883-8ccd-107c566a45d6","Type":"ContainerDied","Data":"d74405c79cd7631f8f26e3673b618b643f4532618091b00f5cd63d3f09aeb2eb"} Jul 28 17:08:55.113672 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:55.113587 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbzsz" Jul 28 17:08:55.114128 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:55.113712 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6dbsb" Jul 28 17:08:55.114128 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:55.113719 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fbzsz" podUID="9baa058e-a765-4c65-b5a6-ce0a7c022d97" Jul 28 17:08:55.114128 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:55.113721 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tdlw" Jul 28 17:08:55.114128 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:55.113816 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6dbsb" podUID="980a4184-ed46-42a1-99f3-86c49cfba43a" Jul 28 17:08:55.114128 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:55.113909 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5tdlw" podUID="93b6b3b5-e860-4fee-9800-0f4534b72e67" Jul 28 17:08:55.571351 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:55.571313 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93b6b3b5-e860-4fee-9800-0f4534b72e67-metrics-certs\") pod \"network-metrics-daemon-5tdlw\" (UID: \"93b6b3b5-e860-4fee-9800-0f4534b72e67\") " pod="openshift-multus/network-metrics-daemon-5tdlw" Jul 28 17:08:55.571551 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:55.571468 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:08:55.571551 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:55.571542 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93b6b3b5-e860-4fee-9800-0f4534b72e67-metrics-certs podName:93b6b3b5-e860-4fee-9800-0f4534b72e67 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:27.571522769 +0000 UTC m=+66.180454117 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/93b6b3b5-e860-4fee-9800-0f4534b72e67-metrics-certs") pod "network-metrics-daemon-5tdlw" (UID: "93b6b3b5-e860-4fee-9800-0f4534b72e67") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:08:55.671895 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:55.671864 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9baa058e-a765-4c65-b5a6-ce0a7c022d97-original-pull-secret\") pod \"global-pull-secret-syncer-fbzsz\" (UID: \"9baa058e-a765-4c65-b5a6-ce0a7c022d97\") " pod="kube-system/global-pull-secret-syncer-fbzsz" Jul 28 17:08:55.671895 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:55.671901 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lzzz\" (UniqueName: \"kubernetes.io/projected/980a4184-ed46-42a1-99f3-86c49cfba43a-kube-api-access-7lzzz\") pod \"network-check-target-6dbsb\" (UID: \"980a4184-ed46-42a1-99f3-86c49cfba43a\") " pod="openshift-network-diagnostics/network-check-target-6dbsb" Jul 28 17:08:55.672092 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:55.672002 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:08:55.672092 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:55.672015 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:08:55.672092 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:55.672018 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:08:55.672092 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:55.672078 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9baa058e-a765-4c65-b5a6-ce0a7c022d97-original-pull-secret podName:9baa058e-a765-4c65-b5a6-ce0a7c022d97 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:27.672058949 +0000 UTC m=+66.280990299 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9baa058e-a765-4c65-b5a6-ce0a7c022d97-original-pull-secret") pod "global-pull-secret-syncer-fbzsz" (UID: "9baa058e-a765-4c65-b5a6-ce0a7c022d97") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:08:55.672297 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:55.672025 2570 projected.go:196] Error preparing data for projected volume kube-api-access-7lzzz for pod openshift-network-diagnostics/network-check-target-6dbsb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:08:55.672297 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:55.672181 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/980a4184-ed46-42a1-99f3-86c49cfba43a-kube-api-access-7lzzz podName:980a4184-ed46-42a1-99f3-86c49cfba43a nodeName:}" failed. No retries permitted until 2026-07-28 17:09:27.672149607 +0000 UTC m=+66.281080974 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-7lzzz" (UniqueName: "kubernetes.io/projected/980a4184-ed46-42a1-99f3-86c49cfba43a-kube-api-access-7lzzz") pod "network-check-target-6dbsb" (UID: "980a4184-ed46-42a1-99f3-86c49cfba43a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:08:56.664901 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.664874 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeReady" Jul 28 17:08:56.665455 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.665046 2570 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 28 17:08:56.701280 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.701249 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-slfqf"] Jul 28 17:08:56.707081 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.706890 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-ngj99"] Jul 28 17:08:56.707257 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.707191 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-slfqf" Jul 28 17:08:56.710666 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.710640 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 28 17:08:56.710792 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.710647 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 28 17:08:56.711052 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.711034 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-vznd7"] Jul 28 17:08:56.711161 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.711138 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 28 17:08:56.711784 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.711338 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 28 17:08:56.711784 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.711418 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-xqxkd\"" Jul 28 17:08:56.711784 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.711693 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-ngj99" Jul 28 17:08:56.714517 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.714482 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 28 17:08:56.714517 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.714505 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 28 17:08:56.714870 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.714851 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-765pv"] Jul 28 17:08:56.715832 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.715662 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-d5n6c\"" Jul 28 17:08:56.716236 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.716210 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-vznd7" Jul 28 17:08:56.723413 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.722797 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 28 17:08:56.723413 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.722876 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 28 17:08:56.723413 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.723051 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 28 17:08:56.723413 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.723305 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 28 17:08:56.724877 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.723887 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 28 17:08:56.724877 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.723890 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-bvjnc\"" Jul 28 17:08:56.724877 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.724444 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-bpkxs"] Jul 28 17:08:56.724877 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.724668 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-765pv" Jul 28 17:08:56.728972 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.728949 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 28 17:08:56.729064 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.729012 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 28 17:08:56.729146 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.728949 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-bpwhm\"" Jul 28 17:08:56.729746 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.729727 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6b5f57f94b-vgwb7"] Jul 28 17:08:56.730871 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.730208 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-bpkxs" Jul 28 17:08:56.733473 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.733454 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 28 17:08:56.733795 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.733732 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 28 17:08:56.734004 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.733983 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 28 17:08:56.734207 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.734190 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-sbz9k"] Jul 28 17:08:56.734479 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.734223 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 28 17:08:56.735415 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.734804 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" Jul 28 17:08:56.735630 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.735611 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-6jpkk\"" Jul 28 17:08:56.740728 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.739200 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 28 17:08:56.740728 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.739279 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-tl7rc"] Jul 28 17:08:56.740728 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.740007 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-sbz9k" Jul 28 17:08:56.740728 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.740528 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6dhdb\"" Jul 28 17:08:56.741032 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.740992 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 28 17:08:56.741866 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.741623 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 28 17:08:56.741866 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.741768 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 28 17:08:56.742710 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.742673 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-m4bxm"] Jul 28 17:08:56.742926 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.742908 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-tl7rc" Jul 28 17:08:56.744514 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.744391 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 28 17:08:56.744998 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.744977 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 28 17:08:56.745239 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.745147 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-hdkhn\"" Jul 28 17:08:56.745239 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.745172 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 28 17:08:56.745943 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.745921 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-9x4tn\"" Jul 28 17:08:56.746084 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.745952 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 28 17:08:56.746294 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.746274 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 28 17:08:56.746408 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.746391 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 28 17:08:56.746663 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.746646 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 28 17:08:56.747101 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.747082 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-64c477cb5c-9stkr"] Jul 28 17:08:56.747308 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.747289 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-m4bxm" Jul 28 17:08:56.747432 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.747311 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 28 17:08:56.752615 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.752228 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-5c7bt\"" Jul 28 17:08:56.752615 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.752517 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 28 17:08:56.752883 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.752523 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 28 17:08:56.752883 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.752652 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 28 17:08:56.753307 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.753287 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-slfqf"] Jul 28 17:08:56.753402 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.753336 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-xx7xr"] Jul 28 17:08:56.757642 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.757626 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 28 17:08:56.758647 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.758625 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-64c477cb5c-9stkr" Jul 28 17:08:56.761983 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.761967 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 28 17:08:56.762413 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.762393 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 28 17:08:56.762582 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.762440 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-c6hl9\"" Jul 28 17:08:56.762686 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.762440 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 28 17:08:56.762924 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.762904 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-765pv"] Jul 28 17:08:56.763022 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.762941 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-bpkxs"] Jul 28 17:08:56.763022 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.762964 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-s9t99"] Jul 28 17:08:56.763022 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.762992 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 28 17:08:56.763022 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.763020 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-xx7xr" Jul 28 17:08:56.763265 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.763076 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 28 17:08:56.763580 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.763554 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 28 17:08:56.765876 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.765860 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-tzq4j\"" Jul 28 17:08:56.766233 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.766211 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-ngj99"] Jul 28 17:08:56.766233 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.766218 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 28 17:08:56.766393 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.766282 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-s9t99" Jul 28 17:08:56.766393 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.766306 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 28 17:08:56.766493 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.766421 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-tl7rc"] Jul 28 17:08:56.766493 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.766452 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-sbz9k"] Jul 28 17:08:56.766493 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.766476 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-64c477cb5c-9stkr"] Jul 28 17:08:56.766625 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.766498 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-49xwx"] Jul 28 17:08:56.769066 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.769047 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-4m279\"" Jul 28 17:08:56.769174 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.769159 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 28 17:08:56.769238 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.769224 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 28 17:08:56.769737 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.769719 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-vznd7"] Jul 28 17:08:56.769828 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.769749 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-xx7xr"] Jul 28 17:08:56.769828 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.769768 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-s9t99"] Jul 28 17:08:56.769828 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.769785 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-m4bxm"] Jul 28 17:08:56.769828 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.769802 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-49xwx"] Jul 28 17:08:56.769828 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.769824 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b5f57f94b-vgwb7"] Jul 28 17:08:56.770041 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.769876 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-49xwx" Jul 28 17:08:56.770041 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.769941 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 28 17:08:56.772712 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.772693 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 28 17:08:56.772807 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.772720 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 28 17:08:56.772807 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.772773 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pz4fk\"" Jul 28 17:08:56.881574 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.881505 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f9b8ad8-774d-4ab8-9826-94eb5a37743f-service-ca-bundle\") pod \"insights-operator-5885fdb759-bpkxs\" (UID: \"8f9b8ad8-774d-4ab8-9826-94eb5a37743f\") " pod="openshift-insights/insights-operator-5885fdb759-bpkxs" Jul 28 17:08:56.881574 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.881548 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lc5z8\" (UniqueName: \"kubernetes.io/projected/ecb82800-8cf0-4ef9-926c-155172fad7ff-kube-api-access-lc5z8\") pod \"console-operator-584dcb6b54-slfqf\" (UID: \"ecb82800-8cf0-4ef9-926c-155172fad7ff\") " pod="openshift-console-operator/console-operator-584dcb6b54-slfqf" Jul 28 17:08:56.881574 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.881574 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnng5\" (UniqueName: \"kubernetes.io/projected/af095ddd-e1cd-4148-9796-d60be4fef0e2-kube-api-access-nnng5\") pod \"router-default-64c477cb5c-9stkr\" (UID: \"af095ddd-e1cd-4148-9796-d60be4fef0e2\") " pod="openshift-ingress/router-default-64c477cb5c-9stkr" Jul 28 17:08:56.881818 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.881599 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2lsk\" (UniqueName: \"kubernetes.io/projected/78838e40-d32b-4f26-9e51-3dd3352026c3-kube-api-access-t2lsk\") pod \"ingress-canary-s9t99\" (UID: \"78838e40-d32b-4f26-9e51-3dd3352026c3\") " pod="openshift-ingress-canary/ingress-canary-s9t99" Jul 28 17:08:56.881818 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.881637 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-registry-tls\") pod \"image-registry-6b5f57f94b-vgwb7\" (UID: \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\") " pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" Jul 28 17:08:56.881818 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.881688 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af095ddd-e1cd-4148-9796-d60be4fef0e2-metrics-certs\") pod \"router-default-64c477cb5c-9stkr\" (UID: \"af095ddd-e1cd-4148-9796-d60be4fef0e2\") " pod="openshift-ingress/router-default-64c477cb5c-9stkr" Jul 28 17:08:56.881818 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.881751 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/78838e40-d32b-4f26-9e51-3dd3352026c3-cert\") pod \"ingress-canary-s9t99\" (UID: \"78838e40-d32b-4f26-9e51-3dd3352026c3\") " pod="openshift-ingress-canary/ingress-canary-s9t99" Jul 28 17:08:56.881818 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.881780 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-trusted-ca\") pod \"image-registry-6b5f57f94b-vgwb7\" (UID: \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\") " pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" Jul 28 17:08:56.881818 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.881815 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjcnd\" (UniqueName: \"kubernetes.io/projected/551cb4c6-f787-4677-b62a-c658a20782d6-kube-api-access-sjcnd\") pod \"network-check-source-7d88cb55d8-xx7xr\" (UID: \"551cb4c6-f787-4677-b62a-c658a20782d6\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-xx7xr" Jul 28 17:08:56.882067 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.881840 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af095ddd-e1cd-4148-9796-d60be4fef0e2-service-ca-bundle\") pod \"router-default-64c477cb5c-9stkr\" (UID: \"af095ddd-e1cd-4148-9796-d60be4fef0e2\") " pod="openshift-ingress/router-default-64c477cb5c-9stkr" Jul 28 17:08:56.882067 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.881876 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fd91b2e7-ff77-474f-a69b-47a84fe174a3-config-volume\") pod \"dns-default-49xwx\" (UID: \"fd91b2e7-ff77-474f-a69b-47a84fe174a3\") " pod="openshift-dns/dns-default-49xwx" Jul 28 17:08:56.882067 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.881927 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-bound-sa-token\") pod \"image-registry-6b5f57f94b-vgwb7\" (UID: \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\") " pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" Jul 28 17:08:56.882067 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.881948 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb5065a0-5a41-40db-a798-5ccd320bdefe-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-sbz9k\" (UID: \"fb5065a0-5a41-40db-a798-5ccd320bdefe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-sbz9k" Jul 28 17:08:56.882067 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.881965 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ecb82800-8cf0-4ef9-926c-155172fad7ff-trusted-ca\") pod \"console-operator-584dcb6b54-slfqf\" (UID: \"ecb82800-8cf0-4ef9-926c-155172fad7ff\") " pod="openshift-console-operator/console-operator-584dcb6b54-slfqf" Jul 28 17:08:56.882067 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.881991 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9a22bd3d-e2c0-461e-b364-b02d99e87eb9-config\") pod \"service-ca-operator-5d6c75f84f-tl7rc\" (UID: \"9a22bd3d-e2c0-461e-b364-b02d99e87eb9\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-tl7rc" Jul 28 17:08:56.882067 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.882027 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f9b8ad8-774d-4ab8-9826-94eb5a37743f-tmp\") pod \"insights-operator-5885fdb759-bpkxs\" (UID: \"8f9b8ad8-774d-4ab8-9826-94eb5a37743f\") " pod="openshift-insights/insights-operator-5885fdb759-bpkxs" Jul 28 17:08:56.882067 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.882054 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glntn\" (UniqueName: \"kubernetes.io/projected/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-kube-api-access-glntn\") pod \"image-registry-6b5f57f94b-vgwb7\" (UID: \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\") " pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" Jul 28 17:08:56.882435 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.882080 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4ddc4d25-529e-44f1-a922-1516040c72f4-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-vznd7\" (UID: \"4ddc4d25-529e-44f1-a922-1516040c72f4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-vznd7" Jul 28 17:08:56.882435 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.882105 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-ca-trust-extracted\") pod \"image-registry-6b5f57f94b-vgwb7\" (UID: \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\") " pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" Jul 28 17:08:56.882435 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.882188 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-installation-pull-secrets\") pod \"image-registry-6b5f57f94b-vgwb7\" (UID: \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\") " pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" Jul 28 17:08:56.882435 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.882212 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/af095ddd-e1cd-4148-9796-d60be4fef0e2-stats-auth\") pod \"router-default-64c477cb5c-9stkr\" (UID: \"af095ddd-e1cd-4148-9796-d60be4fef0e2\") " pod="openshift-ingress/router-default-64c477cb5c-9stkr" Jul 28 17:08:56.882435 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.882271 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8f9b8ad8-774d-4ab8-9826-94eb5a37743f-snapshots\") pod \"insights-operator-5885fdb759-bpkxs\" (UID: \"8f9b8ad8-774d-4ab8-9826-94eb5a37743f\") " pod="openshift-insights/insights-operator-5885fdb759-bpkxs" Jul 28 17:08:56.882435 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.882307 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c6ad5592-e8fc-41e7-8fcf-caa57c3eae35-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-m4bxm\" (UID: \"c6ad5592-e8fc-41e7-8fcf-caa57c3eae35\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-m4bxm" Jul 28 17:08:56.882435 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.882336 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/af095ddd-e1cd-4148-9796-d60be4fef0e2-default-certificate\") pod \"router-default-64c477cb5c-9stkr\" (UID: \"af095ddd-e1cd-4148-9796-d60be4fef0e2\") " pod="openshift-ingress/router-default-64c477cb5c-9stkr" Jul 28 17:08:56.882435 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.882367 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/52730019-2aad-4136-91b5-1b82eb52e0d1-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-ngj99\" (UID: \"52730019-2aad-4136-91b5-1b82eb52e0d1\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-ngj99" Jul 28 17:08:56.882435 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.882403 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c6ad5592-e8fc-41e7-8fcf-caa57c3eae35-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-m4bxm\" (UID: \"c6ad5592-e8fc-41e7-8fcf-caa57c3eae35\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-m4bxm" Jul 28 17:08:56.882835 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.882436 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbp6w\" (UniqueName: \"kubernetes.io/projected/9a22bd3d-e2c0-461e-b364-b02d99e87eb9-kube-api-access-qbp6w\") pod \"service-ca-operator-5d6c75f84f-tl7rc\" (UID: \"9a22bd3d-e2c0-461e-b364-b02d99e87eb9\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-tl7rc" Jul 28 17:08:56.882835 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.882485 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/52730019-2aad-4136-91b5-1b82eb52e0d1-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-ngj99\" (UID: \"52730019-2aad-4136-91b5-1b82eb52e0d1\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-ngj99" Jul 28 17:08:56.882835 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.882517 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-registry-certificates\") pod \"image-registry-6b5f57f94b-vgwb7\" (UID: \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\") " pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" Jul 28 17:08:56.882835 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.882541 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nh59f\" (UniqueName: \"kubernetes.io/projected/fb5065a0-5a41-40db-a798-5ccd320bdefe-kube-api-access-nh59f\") pod \"cluster-samples-operator-6748489656-sbz9k\" (UID: \"fb5065a0-5a41-40db-a798-5ccd320bdefe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-sbz9k" Jul 28 17:08:56.882835 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.882583 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fd91b2e7-ff77-474f-a69b-47a84fe174a3-metrics-tls\") pod \"dns-default-49xwx\" (UID: \"fd91b2e7-ff77-474f-a69b-47a84fe174a3\") " pod="openshift-dns/dns-default-49xwx" Jul 28 17:08:56.882835 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.882635 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd91b2e7-ff77-474f-a69b-47a84fe174a3-tmp-dir\") pod \"dns-default-49xwx\" (UID: \"fd91b2e7-ff77-474f-a69b-47a84fe174a3\") " pod="openshift-dns/dns-default-49xwx" Jul 28 17:08:56.882835 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.882659 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4ddc4d25-529e-44f1-a922-1516040c72f4-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-vznd7\" (UID: \"4ddc4d25-529e-44f1-a922-1516040c72f4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-vznd7" Jul 28 17:08:56.882835 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.882697 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5flr\" (UniqueName: \"kubernetes.io/projected/8f9b8ad8-774d-4ab8-9826-94eb5a37743f-kube-api-access-t5flr\") pod \"insights-operator-5885fdb759-bpkxs\" (UID: \"8f9b8ad8-774d-4ab8-9826-94eb5a37743f\") " pod="openshift-insights/insights-operator-5885fdb759-bpkxs" Jul 28 17:08:56.882835 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.882746 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h642r\" (UniqueName: \"kubernetes.io/projected/ebeb51c6-5d43-493d-b8bd-e56f9993e032-kube-api-access-h642r\") pod \"volume-data-source-validator-549c6d54cf-765pv\" (UID: \"ebeb51c6-5d43-493d-b8bd-e56f9993e032\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-765pv" Jul 28 17:08:56.882835 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.882772 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f9b8ad8-774d-4ab8-9826-94eb5a37743f-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-bpkxs\" (UID: \"8f9b8ad8-774d-4ab8-9826-94eb5a37743f\") " pod="openshift-insights/insights-operator-5885fdb759-bpkxs" Jul 28 17:08:56.882835 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.882805 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ecb82800-8cf0-4ef9-926c-155172fad7ff-serving-cert\") pod \"console-operator-584dcb6b54-slfqf\" (UID: \"ecb82800-8cf0-4ef9-926c-155172fad7ff\") " pod="openshift-console-operator/console-operator-584dcb6b54-slfqf" Jul 28 17:08:56.883262 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.882858 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8f9b8ad8-774d-4ab8-9826-94eb5a37743f-serving-cert\") pod \"insights-operator-5885fdb759-bpkxs\" (UID: \"8f9b8ad8-774d-4ab8-9826-94eb5a37743f\") " pod="openshift-insights/insights-operator-5885fdb759-bpkxs" Jul 28 17:08:56.883262 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.882879 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-image-registry-private-configuration\") pod \"image-registry-6b5f57f94b-vgwb7\" (UID: \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\") " pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" Jul 28 17:08:56.883262 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.882903 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9a22bd3d-e2c0-461e-b364-b02d99e87eb9-serving-cert\") pod \"service-ca-operator-5d6c75f84f-tl7rc\" (UID: \"9a22bd3d-e2c0-461e-b364-b02d99e87eb9\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-tl7rc" Jul 28 17:08:56.883262 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.882927 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bddz5\" (UniqueName: \"kubernetes.io/projected/c6ad5592-e8fc-41e7-8fcf-caa57c3eae35-kube-api-access-bddz5\") pod \"cluster-monitoring-operator-548b489cb8-m4bxm\" (UID: \"c6ad5592-e8fc-41e7-8fcf-caa57c3eae35\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-m4bxm" Jul 28 17:08:56.883262 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.882966 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmpbl\" (UniqueName: \"kubernetes.io/projected/fd91b2e7-ff77-474f-a69b-47a84fe174a3-kube-api-access-jmpbl\") pod \"dns-default-49xwx\" (UID: \"fd91b2e7-ff77-474f-a69b-47a84fe174a3\") " pod="openshift-dns/dns-default-49xwx" Jul 28 17:08:56.883262 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.883005 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5rbv\" (UniqueName: \"kubernetes.io/projected/4ddc4d25-529e-44f1-a922-1516040c72f4-kube-api-access-p5rbv\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-vznd7\" (UID: \"4ddc4d25-529e-44f1-a922-1516040c72f4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-vznd7" Jul 28 17:08:56.883262 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.883025 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ecb82800-8cf0-4ef9-926c-155172fad7ff-config\") pod \"console-operator-584dcb6b54-slfqf\" (UID: \"ecb82800-8cf0-4ef9-926c-155172fad7ff\") " pod="openshift-console-operator/console-operator-584dcb6b54-slfqf" Jul 28 17:08:56.983831 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.983796 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fd91b2e7-ff77-474f-a69b-47a84fe174a3-config-volume\") pod \"dns-default-49xwx\" (UID: \"fd91b2e7-ff77-474f-a69b-47a84fe174a3\") " pod="openshift-dns/dns-default-49xwx" Jul 28 17:08:56.983995 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.983850 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-bound-sa-token\") pod \"image-registry-6b5f57f94b-vgwb7\" (UID: \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\") " pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" Jul 28 17:08:56.984054 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.983986 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb5065a0-5a41-40db-a798-5ccd320bdefe-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-sbz9k\" (UID: \"fb5065a0-5a41-40db-a798-5ccd320bdefe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-sbz9k" Jul 28 17:08:56.984054 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.984021 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ecb82800-8cf0-4ef9-926c-155172fad7ff-trusted-ca\") pod \"console-operator-584dcb6b54-slfqf\" (UID: \"ecb82800-8cf0-4ef9-926c-155172fad7ff\") " pod="openshift-console-operator/console-operator-584dcb6b54-slfqf" Jul 28 17:08:56.984149 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.984051 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9a22bd3d-e2c0-461e-b364-b02d99e87eb9-config\") pod \"service-ca-operator-5d6c75f84f-tl7rc\" (UID: \"9a22bd3d-e2c0-461e-b364-b02d99e87eb9\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-tl7rc" Jul 28 17:08:56.984149 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.984075 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f9b8ad8-774d-4ab8-9826-94eb5a37743f-tmp\") pod \"insights-operator-5885fdb759-bpkxs\" (UID: \"8f9b8ad8-774d-4ab8-9826-94eb5a37743f\") " pod="openshift-insights/insights-operator-5885fdb759-bpkxs" Jul 28 17:08:56.984149 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.984101 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glntn\" (UniqueName: \"kubernetes.io/projected/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-kube-api-access-glntn\") pod \"image-registry-6b5f57f94b-vgwb7\" (UID: \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\") " pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" Jul 28 17:08:56.984149 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:56.984129 2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 28 17:08:56.984300 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.984146 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4ddc4d25-529e-44f1-a922-1516040c72f4-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-vznd7\" (UID: \"4ddc4d25-529e-44f1-a922-1516040c72f4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-vznd7" Jul 28 17:08:56.984300 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.984171 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-ca-trust-extracted\") pod \"image-registry-6b5f57f94b-vgwb7\" (UID: \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\") " pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" Jul 28 17:08:56.984300 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:56.984196 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb5065a0-5a41-40db-a798-5ccd320bdefe-samples-operator-tls podName:fb5065a0-5a41-40db-a798-5ccd320bdefe nodeName:}" failed. No retries permitted until 2026-07-28 17:08:57.484180181 +0000 UTC m=+36.093111529 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fb5065a0-5a41-40db-a798-5ccd320bdefe-samples-operator-tls") pod "cluster-samples-operator-6748489656-sbz9k" (UID: "fb5065a0-5a41-40db-a798-5ccd320bdefe") : secret "samples-operator-tls" not found Jul 28 17:08:56.984300 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.984231 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-installation-pull-secrets\") pod \"image-registry-6b5f57f94b-vgwb7\" (UID: \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\") " pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" Jul 28 17:08:56.984300 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.984253 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/af095ddd-e1cd-4148-9796-d60be4fef0e2-stats-auth\") pod \"router-default-64c477cb5c-9stkr\" (UID: \"af095ddd-e1cd-4148-9796-d60be4fef0e2\") " pod="openshift-ingress/router-default-64c477cb5c-9stkr" Jul 28 17:08:56.984300 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.984272 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8f9b8ad8-774d-4ab8-9826-94eb5a37743f-snapshots\") pod \"insights-operator-5885fdb759-bpkxs\" (UID: \"8f9b8ad8-774d-4ab8-9826-94eb5a37743f\") " pod="openshift-insights/insights-operator-5885fdb759-bpkxs" Jul 28 17:08:56.984300 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.984296 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c6ad5592-e8fc-41e7-8fcf-caa57c3eae35-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-m4bxm\" (UID: \"c6ad5592-e8fc-41e7-8fcf-caa57c3eae35\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-m4bxm" Jul 28 17:08:56.984549 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.984313 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/af095ddd-e1cd-4148-9796-d60be4fef0e2-default-certificate\") pod \"router-default-64c477cb5c-9stkr\" (UID: \"af095ddd-e1cd-4148-9796-d60be4fef0e2\") " pod="openshift-ingress/router-default-64c477cb5c-9stkr" Jul 28 17:08:56.984549 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.984340 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/52730019-2aad-4136-91b5-1b82eb52e0d1-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-ngj99\" (UID: \"52730019-2aad-4136-91b5-1b82eb52e0d1\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-ngj99" Jul 28 17:08:56.984549 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.984365 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c6ad5592-e8fc-41e7-8fcf-caa57c3eae35-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-m4bxm\" (UID: \"c6ad5592-e8fc-41e7-8fcf-caa57c3eae35\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-m4bxm" Jul 28 17:08:56.984549 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.984386 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbp6w\" (UniqueName: \"kubernetes.io/projected/9a22bd3d-e2c0-461e-b364-b02d99e87eb9-kube-api-access-qbp6w\") pod \"service-ca-operator-5d6c75f84f-tl7rc\" (UID: \"9a22bd3d-e2c0-461e-b364-b02d99e87eb9\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-tl7rc" Jul 28 17:08:56.984549 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.984417 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/52730019-2aad-4136-91b5-1b82eb52e0d1-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-ngj99\" (UID: \"52730019-2aad-4136-91b5-1b82eb52e0d1\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-ngj99" Jul 28 17:08:56.984549 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.984432 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-registry-certificates\") pod \"image-registry-6b5f57f94b-vgwb7\" (UID: \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\") " pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" Jul 28 17:08:56.984549 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.984456 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nh59f\" (UniqueName: \"kubernetes.io/projected/fb5065a0-5a41-40db-a798-5ccd320bdefe-kube-api-access-nh59f\") pod \"cluster-samples-operator-6748489656-sbz9k\" (UID: \"fb5065a0-5a41-40db-a798-5ccd320bdefe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-sbz9k" Jul 28 17:08:56.984549 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.984473 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fd91b2e7-ff77-474f-a69b-47a84fe174a3-metrics-tls\") pod \"dns-default-49xwx\" (UID: \"fd91b2e7-ff77-474f-a69b-47a84fe174a3\") " pod="openshift-dns/dns-default-49xwx" Jul 28 17:08:56.984549 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.984489 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd91b2e7-ff77-474f-a69b-47a84fe174a3-tmp-dir\") pod \"dns-default-49xwx\" (UID: \"fd91b2e7-ff77-474f-a69b-47a84fe174a3\") " pod="openshift-dns/dns-default-49xwx" Jul 28 17:08:56.984549 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.984499 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-ca-trust-extracted\") pod \"image-registry-6b5f57f94b-vgwb7\" (UID: \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\") " pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" Jul 28 17:08:56.984549 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.984511 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4ddc4d25-529e-44f1-a922-1516040c72f4-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-vznd7\" (UID: \"4ddc4d25-529e-44f1-a922-1516040c72f4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-vznd7" Jul 28 17:08:56.984549 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.984514 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fd91b2e7-ff77-474f-a69b-47a84fe174a3-config-volume\") pod \"dns-default-49xwx\" (UID: \"fd91b2e7-ff77-474f-a69b-47a84fe174a3\") " pod="openshift-dns/dns-default-49xwx" Jul 28 17:08:56.984549 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.984532 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5flr\" (UniqueName: \"kubernetes.io/projected/8f9b8ad8-774d-4ab8-9826-94eb5a37743f-kube-api-access-t5flr\") pod \"insights-operator-5885fdb759-bpkxs\" (UID: \"8f9b8ad8-774d-4ab8-9826-94eb5a37743f\") " pod="openshift-insights/insights-operator-5885fdb759-bpkxs" Jul 28 17:08:56.985225 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.984567 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h642r\" (UniqueName: \"kubernetes.io/projected/ebeb51c6-5d43-493d-b8bd-e56f9993e032-kube-api-access-h642r\") pod \"volume-data-source-validator-549c6d54cf-765pv\" (UID: \"ebeb51c6-5d43-493d-b8bd-e56f9993e032\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-765pv" Jul 28 17:08:56.985225 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.984582 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f9b8ad8-774d-4ab8-9826-94eb5a37743f-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-bpkxs\" (UID: \"8f9b8ad8-774d-4ab8-9826-94eb5a37743f\") " pod="openshift-insights/insights-operator-5885fdb759-bpkxs" Jul 28 17:08:56.985225 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.984597 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ecb82800-8cf0-4ef9-926c-155172fad7ff-serving-cert\") pod \"console-operator-584dcb6b54-slfqf\" (UID: \"ecb82800-8cf0-4ef9-926c-155172fad7ff\") " pod="openshift-console-operator/console-operator-584dcb6b54-slfqf" Jul 28 17:08:56.985225 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.984620 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8f9b8ad8-774d-4ab8-9826-94eb5a37743f-serving-cert\") pod \"insights-operator-5885fdb759-bpkxs\" (UID: \"8f9b8ad8-774d-4ab8-9826-94eb5a37743f\") " pod="openshift-insights/insights-operator-5885fdb759-bpkxs" Jul 28 17:08:56.985225 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.984643 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-image-registry-private-configuration\") pod \"image-registry-6b5f57f94b-vgwb7\" (UID: \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\") " pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" Jul 28 17:08:56.985225 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.984671 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9a22bd3d-e2c0-461e-b364-b02d99e87eb9-serving-cert\") pod \"service-ca-operator-5d6c75f84f-tl7rc\" (UID: \"9a22bd3d-e2c0-461e-b364-b02d99e87eb9\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-tl7rc" Jul 28 17:08:56.985225 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.984690 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bddz5\" (UniqueName: \"kubernetes.io/projected/c6ad5592-e8fc-41e7-8fcf-caa57c3eae35-kube-api-access-bddz5\") pod \"cluster-monitoring-operator-548b489cb8-m4bxm\" (UID: \"c6ad5592-e8fc-41e7-8fcf-caa57c3eae35\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-m4bxm" Jul 28 17:08:56.985225 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.984712 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmpbl\" (UniqueName: \"kubernetes.io/projected/fd91b2e7-ff77-474f-a69b-47a84fe174a3-kube-api-access-jmpbl\") pod \"dns-default-49xwx\" (UID: \"fd91b2e7-ff77-474f-a69b-47a84fe174a3\") " pod="openshift-dns/dns-default-49xwx" Jul 28 17:08:56.985225 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.984727 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5rbv\" (UniqueName: \"kubernetes.io/projected/4ddc4d25-529e-44f1-a922-1516040c72f4-kube-api-access-p5rbv\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-vznd7\" (UID: \"4ddc4d25-529e-44f1-a922-1516040c72f4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-vznd7" Jul 28 17:08:56.985225 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.984744 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ecb82800-8cf0-4ef9-926c-155172fad7ff-config\") pod \"console-operator-584dcb6b54-slfqf\" (UID: \"ecb82800-8cf0-4ef9-926c-155172fad7ff\") " pod="openshift-console-operator/console-operator-584dcb6b54-slfqf" Jul 28 17:08:56.985225 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:56.984754 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 17:08:56.985225 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.984772 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f9b8ad8-774d-4ab8-9826-94eb5a37743f-service-ca-bundle\") pod \"insights-operator-5885fdb759-bpkxs\" (UID: \"8f9b8ad8-774d-4ab8-9826-94eb5a37743f\") " pod="openshift-insights/insights-operator-5885fdb759-bpkxs" Jul 28 17:08:56.985225 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.984798 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lc5z8\" (UniqueName: \"kubernetes.io/projected/ecb82800-8cf0-4ef9-926c-155172fad7ff-kube-api-access-lc5z8\") pod \"console-operator-584dcb6b54-slfqf\" (UID: \"ecb82800-8cf0-4ef9-926c-155172fad7ff\") " pod="openshift-console-operator/console-operator-584dcb6b54-slfqf" Jul 28 17:08:56.985225 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:56.984820 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6ad5592-e8fc-41e7-8fcf-caa57c3eae35-cluster-monitoring-operator-tls podName:c6ad5592-e8fc-41e7-8fcf-caa57c3eae35 nodeName:}" failed. No retries permitted until 2026-07-28 17:08:57.48480397 +0000 UTC m=+36.093735321 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c6ad5592-e8fc-41e7-8fcf-caa57c3eae35-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-m4bxm" (UID: "c6ad5592-e8fc-41e7-8fcf-caa57c3eae35") : secret "cluster-monitoring-operator-tls" not found Jul 28 17:08:56.985225 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.984878 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnng5\" (UniqueName: \"kubernetes.io/projected/af095ddd-e1cd-4148-9796-d60be4fef0e2-kube-api-access-nnng5\") pod \"router-default-64c477cb5c-9stkr\" (UID: \"af095ddd-e1cd-4148-9796-d60be4fef0e2\") " pod="openshift-ingress/router-default-64c477cb5c-9stkr" Jul 28 17:08:56.985930 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.984916 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2lsk\" (UniqueName: \"kubernetes.io/projected/78838e40-d32b-4f26-9e51-3dd3352026c3-kube-api-access-t2lsk\") pod \"ingress-canary-s9t99\" (UID: \"78838e40-d32b-4f26-9e51-3dd3352026c3\") " pod="openshift-ingress-canary/ingress-canary-s9t99" Jul 28 17:08:56.985930 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.984959 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-registry-tls\") pod \"image-registry-6b5f57f94b-vgwb7\" (UID: \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\") " pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" Jul 28 17:08:56.985930 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:56.985056 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 17:08:56.985930 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:56.985068 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b5f57f94b-vgwb7: secret "image-registry-tls" not found Jul 28 17:08:56.985930 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:56.985105 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-registry-tls podName:2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c nodeName:}" failed. No retries permitted until 2026-07-28 17:08:57.485091717 +0000 UTC m=+36.094023070 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-registry-tls") pod "image-registry-6b5f57f94b-vgwb7" (UID: "2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c") : secret "image-registry-tls" not found Jul 28 17:08:56.985930 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.985064 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9a22bd3d-e2c0-461e-b364-b02d99e87eb9-config\") pod \"service-ca-operator-5d6c75f84f-tl7rc\" (UID: \"9a22bd3d-e2c0-461e-b364-b02d99e87eb9\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-tl7rc" Jul 28 17:08:56.985930 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.985168 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c6ad5592-e8fc-41e7-8fcf-caa57c3eae35-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-m4bxm\" (UID: \"c6ad5592-e8fc-41e7-8fcf-caa57c3eae35\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-m4bxm" Jul 28 17:08:56.985930 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:56.985253 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 28 17:08:56.985930 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:56.985299 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52730019-2aad-4136-91b5-1b82eb52e0d1-networking-console-plugin-cert podName:52730019-2aad-4136-91b5-1b82eb52e0d1 nodeName:}" failed. No retries permitted until 2026-07-28 17:08:57.485284894 +0000 UTC m=+36.094216243 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/52730019-2aad-4136-91b5-1b82eb52e0d1-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-ngj99" (UID: "52730019-2aad-4136-91b5-1b82eb52e0d1") : secret "networking-console-plugin-cert" not found Jul 28 17:08:56.985930 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.985295 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f9b8ad8-774d-4ab8-9826-94eb5a37743f-tmp\") pod \"insights-operator-5885fdb759-bpkxs\" (UID: \"8f9b8ad8-774d-4ab8-9826-94eb5a37743f\") " pod="openshift-insights/insights-operator-5885fdb759-bpkxs" Jul 28 17:08:56.985930 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.985251 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8f9b8ad8-774d-4ab8-9826-94eb5a37743f-snapshots\") pod \"insights-operator-5885fdb759-bpkxs\" (UID: \"8f9b8ad8-774d-4ab8-9826-94eb5a37743f\") " pod="openshift-insights/insights-operator-5885fdb759-bpkxs" Jul 28 17:08:56.985930 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.985536 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ecb82800-8cf0-4ef9-926c-155172fad7ff-trusted-ca\") pod \"console-operator-584dcb6b54-slfqf\" (UID: \"ecb82800-8cf0-4ef9-926c-155172fad7ff\") " pod="openshift-console-operator/console-operator-584dcb6b54-slfqf" Jul 28 17:08:56.985930 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.985801 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/52730019-2aad-4136-91b5-1b82eb52e0d1-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-ngj99\" (UID: \"52730019-2aad-4136-91b5-1b82eb52e0d1\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-ngj99" Jul 28 17:08:56.986557 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.986032 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4ddc4d25-529e-44f1-a922-1516040c72f4-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-vznd7\" (UID: \"4ddc4d25-529e-44f1-a922-1516040c72f4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-vznd7" Jul 28 17:08:56.986819 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.986639 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ecb82800-8cf0-4ef9-926c-155172fad7ff-config\") pod \"console-operator-584dcb6b54-slfqf\" (UID: \"ecb82800-8cf0-4ef9-926c-155172fad7ff\") " pod="openshift-console-operator/console-operator-584dcb6b54-slfqf" Jul 28 17:08:56.986819 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:56.986775 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:08:56.989796 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.987816 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f9b8ad8-774d-4ab8-9826-94eb5a37743f-service-ca-bundle\") pod \"insights-operator-5885fdb759-bpkxs\" (UID: \"8f9b8ad8-774d-4ab8-9826-94eb5a37743f\") " pod="openshift-insights/insights-operator-5885fdb759-bpkxs" Jul 28 17:08:56.989796 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:56.987825 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd91b2e7-ff77-474f-a69b-47a84fe174a3-metrics-tls podName:fd91b2e7-ff77-474f-a69b-47a84fe174a3 nodeName:}" failed. No retries permitted until 2026-07-28 17:08:57.487808208 +0000 UTC m=+36.096739558 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fd91b2e7-ff77-474f-a69b-47a84fe174a3-metrics-tls") pod "dns-default-49xwx" (UID: "fd91b2e7-ff77-474f-a69b-47a84fe174a3") : secret "dns-default-metrics-tls" not found Jul 28 17:08:56.989796 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.987893 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af095ddd-e1cd-4148-9796-d60be4fef0e2-metrics-certs\") pod \"router-default-64c477cb5c-9stkr\" (UID: \"af095ddd-e1cd-4148-9796-d60be4fef0e2\") " pod="openshift-ingress/router-default-64c477cb5c-9stkr" Jul 28 17:08:56.989796 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.987938 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/78838e40-d32b-4f26-9e51-3dd3352026c3-cert\") pod \"ingress-canary-s9t99\" (UID: \"78838e40-d32b-4f26-9e51-3dd3352026c3\") " pod="openshift-ingress-canary/ingress-canary-s9t99" Jul 28 17:08:56.989796 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.987965 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-trusted-ca\") pod \"image-registry-6b5f57f94b-vgwb7\" (UID: \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\") " pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" Jul 28 17:08:56.989796 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.988003 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjcnd\" (UniqueName: \"kubernetes.io/projected/551cb4c6-f787-4677-b62a-c658a20782d6-kube-api-access-sjcnd\") pod \"network-check-source-7d88cb55d8-xx7xr\" (UID: \"551cb4c6-f787-4677-b62a-c658a20782d6\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-xx7xr" Jul 28 17:08:56.989796 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.988038 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af095ddd-e1cd-4148-9796-d60be4fef0e2-service-ca-bundle\") pod \"router-default-64c477cb5c-9stkr\" (UID: \"af095ddd-e1cd-4148-9796-d60be4fef0e2\") " pod="openshift-ingress/router-default-64c477cb5c-9stkr" Jul 28 17:08:56.989796 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:56.988421 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/af095ddd-e1cd-4148-9796-d60be4fef0e2-service-ca-bundle podName:af095ddd-e1cd-4148-9796-d60be4fef0e2 nodeName:}" failed. No retries permitted until 2026-07-28 17:08:57.488404569 +0000 UTC m=+36.097335940 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/af095ddd-e1cd-4148-9796-d60be4fef0e2-service-ca-bundle") pod "router-default-64c477cb5c-9stkr" (UID: "af095ddd-e1cd-4148-9796-d60be4fef0e2") : configmap references non-existent config key: service-ca.crt Jul 28 17:08:56.989796 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:56.988521 2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 28 17:08:56.989796 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:56.988571 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af095ddd-e1cd-4148-9796-d60be4fef0e2-metrics-certs podName:af095ddd-e1cd-4148-9796-d60be4fef0e2 nodeName:}" failed. No retries permitted until 2026-07-28 17:08:57.48855129 +0000 UTC m=+36.097482640 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af095ddd-e1cd-4148-9796-d60be4fef0e2-metrics-certs") pod "router-default-64c477cb5c-9stkr" (UID: "af095ddd-e1cd-4148-9796-d60be4fef0e2") : secret "router-metrics-certs-default" not found Jul 28 17:08:56.989796 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.989162 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd91b2e7-ff77-474f-a69b-47a84fe174a3-tmp-dir\") pod \"dns-default-49xwx\" (UID: \"fd91b2e7-ff77-474f-a69b-47a84fe174a3\") " pod="openshift-dns/dns-default-49xwx" Jul 28 17:08:56.989796 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.989341 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-registry-certificates\") pod \"image-registry-6b5f57f94b-vgwb7\" (UID: \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\") " pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" Jul 28 17:08:56.990386 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.990299 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f9b8ad8-774d-4ab8-9826-94eb5a37743f-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-bpkxs\" (UID: \"8f9b8ad8-774d-4ab8-9826-94eb5a37743f\") " pod="openshift-insights/insights-operator-5885fdb759-bpkxs" Jul 28 17:08:56.990663 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.990641 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-installation-pull-secrets\") pod \"image-registry-6b5f57f94b-vgwb7\" (UID: \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\") " pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" Jul 28 17:08:56.991773 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.991751 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8f9b8ad8-774d-4ab8-9826-94eb5a37743f-serving-cert\") pod \"insights-operator-5885fdb759-bpkxs\" (UID: \"8f9b8ad8-774d-4ab8-9826-94eb5a37743f\") " pod="openshift-insights/insights-operator-5885fdb759-bpkxs" Jul 28 17:08:56.992131 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.992089 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4ddc4d25-529e-44f1-a922-1516040c72f4-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-vznd7\" (UID: \"4ddc4d25-529e-44f1-a922-1516040c72f4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-vznd7" Jul 28 17:08:56.992614 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.992591 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-trusted-ca\") pod \"image-registry-6b5f57f94b-vgwb7\" (UID: \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\") " pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" Jul 28 17:08:56.993838 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.993131 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/af095ddd-e1cd-4148-9796-d60be4fef0e2-stats-auth\") pod \"router-default-64c477cb5c-9stkr\" (UID: \"af095ddd-e1cd-4148-9796-d60be4fef0e2\") " pod="openshift-ingress/router-default-64c477cb5c-9stkr" Jul 28 17:08:56.993838 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.993548 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/af095ddd-e1cd-4148-9796-d60be4fef0e2-default-certificate\") pod \"router-default-64c477cb5c-9stkr\" (UID: \"af095ddd-e1cd-4148-9796-d60be4fef0e2\") " pod="openshift-ingress/router-default-64c477cb5c-9stkr" Jul 28 17:08:56.994020 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.994001 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-image-registry-private-configuration\") pod \"image-registry-6b5f57f94b-vgwb7\" (UID: \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\") " pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" Jul 28 17:08:57.000038 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.998811 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbp6w\" (UniqueName: \"kubernetes.io/projected/9a22bd3d-e2c0-461e-b364-b02d99e87eb9-kube-api-access-qbp6w\") pod \"service-ca-operator-5d6c75f84f-tl7rc\" (UID: \"9a22bd3d-e2c0-461e-b364-b02d99e87eb9\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-tl7rc" Jul 28 17:08:57.000038 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.999064 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ecb82800-8cf0-4ef9-926c-155172fad7ff-serving-cert\") pod \"console-operator-584dcb6b54-slfqf\" (UID: \"ecb82800-8cf0-4ef9-926c-155172fad7ff\") " pod="openshift-console-operator/console-operator-584dcb6b54-slfqf" Jul 28 17:08:57.000038 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.999414 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nh59f\" (UniqueName: \"kubernetes.io/projected/fb5065a0-5a41-40db-a798-5ccd320bdefe-kube-api-access-nh59f\") pod \"cluster-samples-operator-6748489656-sbz9k\" (UID: \"fb5065a0-5a41-40db-a798-5ccd320bdefe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-sbz9k" Jul 28 17:08:57.000038 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:56.999510 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmpbl\" (UniqueName: \"kubernetes.io/projected/fd91b2e7-ff77-474f-a69b-47a84fe174a3-kube-api-access-jmpbl\") pod \"dns-default-49xwx\" (UID: \"fd91b2e7-ff77-474f-a69b-47a84fe174a3\") " pod="openshift-dns/dns-default-49xwx" Jul 28 17:08:57.000524 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.000500 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bddz5\" (UniqueName: \"kubernetes.io/projected/c6ad5592-e8fc-41e7-8fcf-caa57c3eae35-kube-api-access-bddz5\") pod \"cluster-monitoring-operator-548b489cb8-m4bxm\" (UID: \"c6ad5592-e8fc-41e7-8fcf-caa57c3eae35\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-m4bxm" Jul 28 17:08:57.001686 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.001444 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2lsk\" (UniqueName: \"kubernetes.io/projected/78838e40-d32b-4f26-9e51-3dd3352026c3-kube-api-access-t2lsk\") pod \"ingress-canary-s9t99\" (UID: \"78838e40-d32b-4f26-9e51-3dd3352026c3\") " pod="openshift-ingress-canary/ingress-canary-s9t99" Jul 28 17:08:57.002189 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.002103 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glntn\" (UniqueName: \"kubernetes.io/projected/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-kube-api-access-glntn\") pod \"image-registry-6b5f57f94b-vgwb7\" (UID: \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\") " pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" Jul 28 17:08:57.004358 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.003250 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-bound-sa-token\") pod \"image-registry-6b5f57f94b-vgwb7\" (UID: \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\") " pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" Jul 28 17:08:57.004358 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.003446 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5flr\" (UniqueName: \"kubernetes.io/projected/8f9b8ad8-774d-4ab8-9826-94eb5a37743f-kube-api-access-t5flr\") pod \"insights-operator-5885fdb759-bpkxs\" (UID: \"8f9b8ad8-774d-4ab8-9826-94eb5a37743f\") " pod="openshift-insights/insights-operator-5885fdb759-bpkxs" Jul 28 17:08:57.004358 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.003523 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9a22bd3d-e2c0-461e-b364-b02d99e87eb9-serving-cert\") pod \"service-ca-operator-5d6c75f84f-tl7rc\" (UID: \"9a22bd3d-e2c0-461e-b364-b02d99e87eb9\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-tl7rc" Jul 28 17:08:57.004358 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.003942 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lc5z8\" (UniqueName: \"kubernetes.io/projected/ecb82800-8cf0-4ef9-926c-155172fad7ff-kube-api-access-lc5z8\") pod \"console-operator-584dcb6b54-slfqf\" (UID: \"ecb82800-8cf0-4ef9-926c-155172fad7ff\") " pod="openshift-console-operator/console-operator-584dcb6b54-slfqf" Jul 28 17:08:57.004358 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.004060 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnng5\" (UniqueName: \"kubernetes.io/projected/af095ddd-e1cd-4148-9796-d60be4fef0e2-kube-api-access-nnng5\") pod \"router-default-64c477cb5c-9stkr\" (UID: \"af095ddd-e1cd-4148-9796-d60be4fef0e2\") " pod="openshift-ingress/router-default-64c477cb5c-9stkr" Jul 28 17:08:57.004358 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.004142 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5rbv\" (UniqueName: \"kubernetes.io/projected/4ddc4d25-529e-44f1-a922-1516040c72f4-kube-api-access-p5rbv\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-vznd7\" (UID: \"4ddc4d25-529e-44f1-a922-1516040c72f4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-vznd7" Jul 28 17:08:57.004358 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.004312 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/78838e40-d32b-4f26-9e51-3dd3352026c3-cert\") pod \"ingress-canary-s9t99\" (UID: \"78838e40-d32b-4f26-9e51-3dd3352026c3\") " pod="openshift-ingress-canary/ingress-canary-s9t99" Jul 28 17:08:57.005369 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.005332 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h642r\" (UniqueName: \"kubernetes.io/projected/ebeb51c6-5d43-493d-b8bd-e56f9993e032-kube-api-access-h642r\") pod \"volume-data-source-validator-549c6d54cf-765pv\" (UID: \"ebeb51c6-5d43-493d-b8bd-e56f9993e032\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-765pv" Jul 28 17:08:57.006671 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.006647 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjcnd\" (UniqueName: \"kubernetes.io/projected/551cb4c6-f787-4677-b62a-c658a20782d6-kube-api-access-sjcnd\") pod \"network-check-source-7d88cb55d8-xx7xr\" (UID: \"551cb4c6-f787-4677-b62a-c658a20782d6\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-xx7xr" Jul 28 17:08:57.030277 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.030241 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-slfqf" Jul 28 17:08:57.048428 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.048403 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-vznd7" Jul 28 17:08:57.059378 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.059354 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-765pv" Jul 28 17:08:57.069300 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.069279 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-bpkxs" Jul 28 17:08:57.090493 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.090473 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-tl7rc" Jul 28 17:08:57.111837 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.111813 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-xx7xr" Jul 28 17:08:57.114064 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.114038 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbzsz" Jul 28 17:08:57.114064 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.114057 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6dbsb" Jul 28 17:08:57.114240 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.114222 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tdlw" Jul 28 17:08:57.117430 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.117398 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 28 17:08:57.117524 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.117479 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8hnfw\"" Jul 28 17:08:57.117524 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.117503 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 28 17:08:57.117637 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.117522 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hr5cl\"" Jul 28 17:08:57.119075 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.119057 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-s9t99" Jul 28 17:08:57.492102 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.491920 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb5065a0-5a41-40db-a798-5ccd320bdefe-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-sbz9k\" (UID: \"fb5065a0-5a41-40db-a798-5ccd320bdefe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-sbz9k" Jul 28 17:08:57.492102 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.492160 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c6ad5592-e8fc-41e7-8fcf-caa57c3eae35-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-m4bxm\" (UID: \"c6ad5592-e8fc-41e7-8fcf-caa57c3eae35\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-m4bxm" Jul 28 17:08:57.492332 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.492195 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/52730019-2aad-4136-91b5-1b82eb52e0d1-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-ngj99\" (UID: \"52730019-2aad-4136-91b5-1b82eb52e0d1\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-ngj99" Jul 28 17:08:57.492332 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:57.492205 2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 28 17:08:57.492332 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.492235 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fd91b2e7-ff77-474f-a69b-47a84fe174a3-metrics-tls\") pod \"dns-default-49xwx\" (UID: \"fd91b2e7-ff77-474f-a69b-47a84fe174a3\") " pod="openshift-dns/dns-default-49xwx" Jul 28 17:08:57.492332 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:57.492300 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb5065a0-5a41-40db-a798-5ccd320bdefe-samples-operator-tls podName:fb5065a0-5a41-40db-a798-5ccd320bdefe nodeName:}" failed. No retries permitted until 2026-07-28 17:08:58.492276804 +0000 UTC m=+37.101208191 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fb5065a0-5a41-40db-a798-5ccd320bdefe-samples-operator-tls") pod "cluster-samples-operator-6748489656-sbz9k" (UID: "fb5065a0-5a41-40db-a798-5ccd320bdefe") : secret "samples-operator-tls" not found Jul 28 17:08:57.492537 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:57.492358 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:08:57.492537 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.492398 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-registry-tls\") pod \"image-registry-6b5f57f94b-vgwb7\" (UID: \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\") " pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" Jul 28 17:08:57.492537 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:57.492410 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd91b2e7-ff77-474f-a69b-47a84fe174a3-metrics-tls podName:fd91b2e7-ff77-474f-a69b-47a84fe174a3 nodeName:}" failed. No retries permitted until 2026-07-28 17:08:58.492393979 +0000 UTC m=+37.101325328 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fd91b2e7-ff77-474f-a69b-47a84fe174a3-metrics-tls") pod "dns-default-49xwx" (UID: "fd91b2e7-ff77-474f-a69b-47a84fe174a3") : secret "dns-default-metrics-tls" not found Jul 28 17:08:57.492537 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.492450 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af095ddd-e1cd-4148-9796-d60be4fef0e2-metrics-certs\") pod \"router-default-64c477cb5c-9stkr\" (UID: \"af095ddd-e1cd-4148-9796-d60be4fef0e2\") " pod="openshift-ingress/router-default-64c477cb5c-9stkr" Jul 28 17:08:57.492537 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:57.492474 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 17:08:57.492537 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:57.492486 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b5f57f94b-vgwb7: secret "image-registry-tls" not found Jul 28 17:08:57.492537 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.492488 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af095ddd-e1cd-4148-9796-d60be4fef0e2-service-ca-bundle\") pod \"router-default-64c477cb5c-9stkr\" (UID: \"af095ddd-e1cd-4148-9796-d60be4fef0e2\") " pod="openshift-ingress/router-default-64c477cb5c-9stkr" Jul 28 17:08:57.492537 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:57.492525 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-registry-tls podName:2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c nodeName:}" failed. No retries permitted until 2026-07-28 17:08:58.492512731 +0000 UTC m=+37.101444098 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-registry-tls") pod "image-registry-6b5f57f94b-vgwb7" (UID: "2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c") : secret "image-registry-tls" not found Jul 28 17:08:57.492893 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:57.492589 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/af095ddd-e1cd-4148-9796-d60be4fef0e2-service-ca-bundle podName:af095ddd-e1cd-4148-9796-d60be4fef0e2 nodeName:}" failed. No retries permitted until 2026-07-28 17:08:58.492578536 +0000 UTC m=+37.101509886 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/af095ddd-e1cd-4148-9796-d60be4fef0e2-service-ca-bundle") pod "router-default-64c477cb5c-9stkr" (UID: "af095ddd-e1cd-4148-9796-d60be4fef0e2") : configmap references non-existent config key: service-ca.crt Jul 28 17:08:57.492893 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:57.492650 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 17:08:57.493171 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:57.493152 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 28 17:08:57.493256 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:57.493244 2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 28 17:08:57.496784 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:57.496764 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6ad5592-e8fc-41e7-8fcf-caa57c3eae35-cluster-monitoring-operator-tls podName:c6ad5592-e8fc-41e7-8fcf-caa57c3eae35 nodeName:}" failed. No retries permitted until 2026-07-28 17:08:58.492674661 +0000 UTC m=+37.101606009 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c6ad5592-e8fc-41e7-8fcf-caa57c3eae35-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-m4bxm" (UID: "c6ad5592-e8fc-41e7-8fcf-caa57c3eae35") : secret "cluster-monitoring-operator-tls" not found Jul 28 17:08:57.496896 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:57.496808 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52730019-2aad-4136-91b5-1b82eb52e0d1-networking-console-plugin-cert podName:52730019-2aad-4136-91b5-1b82eb52e0d1 nodeName:}" failed. No retries permitted until 2026-07-28 17:08:58.496790243 +0000 UTC m=+37.105721598 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/52730019-2aad-4136-91b5-1b82eb52e0d1-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-ngj99" (UID: "52730019-2aad-4136-91b5-1b82eb52e0d1") : secret "networking-console-plugin-cert" not found Jul 28 17:08:57.496896 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:57.496829 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af095ddd-e1cd-4148-9796-d60be4fef0e2-metrics-certs podName:af095ddd-e1cd-4148-9796-d60be4fef0e2 nodeName:}" failed. No retries permitted until 2026-07-28 17:08:58.49681911 +0000 UTC m=+37.105750772 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af095ddd-e1cd-4148-9796-d60be4fef0e2-metrics-certs") pod "router-default-64c477cb5c-9stkr" (UID: "af095ddd-e1cd-4148-9796-d60be4fef0e2") : secret "router-metrics-certs-default" not found Jul 28 17:08:57.664561 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.664333 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-tl7rc"] Jul 28 17:08:57.668277 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:57.668241 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a22bd3d_e2c0_461e_b364_b02d99e87eb9.slice/crio-b863b21288ea9964a56b96eae5073ddcedf3f36386efc4f897c593e6bd32b844 WatchSource:0}: Error finding container b863b21288ea9964a56b96eae5073ddcedf3f36386efc4f897c593e6bd32b844: Status 404 returned error can't find the container with id b863b21288ea9964a56b96eae5073ddcedf3f36386efc4f897c593e6bd32b844 Jul 28 17:08:57.675704 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.675683 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-xx7xr"] Jul 28 17:08:57.678054 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.678006 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-vznd7"] Jul 28 17:08:57.680239 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.680210 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-slfqf"] Jul 28 17:08:57.695503 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.695475 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-765pv"] Jul 28 17:08:57.699970 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.699897 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-s9t99"] Jul 28 17:08:57.702794 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:57.702775 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-bpkxs"] Jul 28 17:08:57.705320 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:57.705264 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebeb51c6_5d43_493d_b8bd_e56f9993e032.slice/crio-98c01b88fdff212dcd7248f715ce9a04fd7655fc2bcbcc5e9a0b191f766af2ec WatchSource:0}: Error finding container 98c01b88fdff212dcd7248f715ce9a04fd7655fc2bcbcc5e9a0b191f766af2ec: Status 404 returned error can't find the container with id 98c01b88fdff212dcd7248f715ce9a04fd7655fc2bcbcc5e9a0b191f766af2ec Jul 28 17:08:57.705839 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:57.705808 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f9b8ad8_774d_4ab8_9826_94eb5a37743f.slice/crio-aa00c2e70c647293b8a7aa051117cf24c349f7a9510ed4e417e444b7c65cac1a WatchSource:0}: Error finding container aa00c2e70c647293b8a7aa051117cf24c349f7a9510ed4e417e444b7c65cac1a: Status 404 returned error can't find the container with id aa00c2e70c647293b8a7aa051117cf24c349f7a9510ed4e417e444b7c65cac1a Jul 28 17:08:57.708347 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:08:57.708258 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78838e40_d32b_4f26_9e51_3dd3352026c3.slice/crio-4fc3e10fb5e0d31b59f5b6f854a9f239ee12fac3de10efdd3efef05622b1b2bc WatchSource:0}: Error finding container 4fc3e10fb5e0d31b59f5b6f854a9f239ee12fac3de10efdd3efef05622b1b2bc: Status 404 returned error can't find the container with id 4fc3e10fb5e0d31b59f5b6f854a9f239ee12fac3de10efdd3efef05622b1b2bc Jul 28 17:08:58.393923 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:58.393885 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-bpkxs" event={"ID":"8f9b8ad8-774d-4ab8-9826-94eb5a37743f","Type":"ContainerStarted","Data":"aa00c2e70c647293b8a7aa051117cf24c349f7a9510ed4e417e444b7c65cac1a"} Jul 28 17:08:58.395572 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:58.395545 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-slfqf" event={"ID":"ecb82800-8cf0-4ef9-926c-155172fad7ff","Type":"ContainerStarted","Data":"7b171c63115786eee902bec77709eacd01ac5a0ce2ac730af0a496e2024ebbe0"} Jul 28 17:08:58.406283 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:58.406258 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-xx7xr" event={"ID":"551cb4c6-f787-4677-b62a-c658a20782d6","Type":"ContainerStarted","Data":"d82b633c294337db871f4dbb4d2677911e6b8844eb5e820eb850cd4ce59151e0"} Jul 28 17:08:58.411628 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:58.411545 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-tl7rc" event={"ID":"9a22bd3d-e2c0-461e-b364-b02d99e87eb9","Type":"ContainerStarted","Data":"b863b21288ea9964a56b96eae5073ddcedf3f36386efc4f897c593e6bd32b844"} Jul 28 17:08:58.422877 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:58.422832 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-765pv" event={"ID":"ebeb51c6-5d43-493d-b8bd-e56f9993e032","Type":"ContainerStarted","Data":"98c01b88fdff212dcd7248f715ce9a04fd7655fc2bcbcc5e9a0b191f766af2ec"} Jul 28 17:08:58.424981 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:58.424917 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-s9t99" event={"ID":"78838e40-d32b-4f26-9e51-3dd3352026c3","Type":"ContainerStarted","Data":"4fc3e10fb5e0d31b59f5b6f854a9f239ee12fac3de10efdd3efef05622b1b2bc"} Jul 28 17:08:58.435857 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:58.434968 2570 generic.go:356] "Generic (PLEG): container finished" podID="36c66dd3-6c06-4883-8ccd-107c566a45d6" containerID="c1add796bfcadc0a1fc2a13a6395a136044eb9058fd626c73cc51e95645218bf" exitCode=0 Jul 28 17:08:58.435857 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:58.435033 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b9mxn" event={"ID":"36c66dd3-6c06-4883-8ccd-107c566a45d6","Type":"ContainerDied","Data":"c1add796bfcadc0a1fc2a13a6395a136044eb9058fd626c73cc51e95645218bf"} Jul 28 17:08:58.446164 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:58.441161 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-vznd7" event={"ID":"4ddc4d25-529e-44f1-a922-1516040c72f4","Type":"ContainerStarted","Data":"56b74780e98dbf998e9f64eb7360caa9b9d16bd9fcf54019663e6edf6f83d680"} Jul 28 17:08:58.503078 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:58.501666 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af095ddd-e1cd-4148-9796-d60be4fef0e2-service-ca-bundle\") pod \"router-default-64c477cb5c-9stkr\" (UID: \"af095ddd-e1cd-4148-9796-d60be4fef0e2\") " pod="openshift-ingress/router-default-64c477cb5c-9stkr" Jul 28 17:08:58.503078 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:58.501737 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb5065a0-5a41-40db-a798-5ccd320bdefe-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-sbz9k\" (UID: \"fb5065a0-5a41-40db-a798-5ccd320bdefe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-sbz9k" Jul 28 17:08:58.503078 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:58.501779 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c6ad5592-e8fc-41e7-8fcf-caa57c3eae35-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-m4bxm\" (UID: \"c6ad5592-e8fc-41e7-8fcf-caa57c3eae35\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-m4bxm" Jul 28 17:08:58.503078 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:58.501807 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/52730019-2aad-4136-91b5-1b82eb52e0d1-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-ngj99\" (UID: \"52730019-2aad-4136-91b5-1b82eb52e0d1\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-ngj99" Jul 28 17:08:58.503078 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:58.501840 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fd91b2e7-ff77-474f-a69b-47a84fe174a3-metrics-tls\") pod \"dns-default-49xwx\" (UID: \"fd91b2e7-ff77-474f-a69b-47a84fe174a3\") " pod="openshift-dns/dns-default-49xwx" Jul 28 17:08:58.503078 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:58.501921 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-registry-tls\") pod \"image-registry-6b5f57f94b-vgwb7\" (UID: \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\") " pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" Jul 28 17:08:58.503078 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:58.501954 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af095ddd-e1cd-4148-9796-d60be4fef0e2-metrics-certs\") pod \"router-default-64c477cb5c-9stkr\" (UID: \"af095ddd-e1cd-4148-9796-d60be4fef0e2\") " pod="openshift-ingress/router-default-64c477cb5c-9stkr" Jul 28 17:08:58.503078 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:58.502079 2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 28 17:08:58.503078 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:58.502156 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af095ddd-e1cd-4148-9796-d60be4fef0e2-metrics-certs podName:af095ddd-e1cd-4148-9796-d60be4fef0e2 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:00.502137305 +0000 UTC m=+39.111068670 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af095ddd-e1cd-4148-9796-d60be4fef0e2-metrics-certs") pod "router-default-64c477cb5c-9stkr" (UID: "af095ddd-e1cd-4148-9796-d60be4fef0e2") : secret "router-metrics-certs-default" not found Jul 28 17:08:58.503078 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:58.502570 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/af095ddd-e1cd-4148-9796-d60be4fef0e2-service-ca-bundle podName:af095ddd-e1cd-4148-9796-d60be4fef0e2 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:00.502551404 +0000 UTC m=+39.111482766 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/af095ddd-e1cd-4148-9796-d60be4fef0e2-service-ca-bundle") pod "router-default-64c477cb5c-9stkr" (UID: "af095ddd-e1cd-4148-9796-d60be4fef0e2") : configmap references non-existent config key: service-ca.crt Jul 28 17:08:58.503078 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:58.502646 2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 28 17:08:58.503078 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:58.502681 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb5065a0-5a41-40db-a798-5ccd320bdefe-samples-operator-tls podName:fb5065a0-5a41-40db-a798-5ccd320bdefe nodeName:}" failed. No retries permitted until 2026-07-28 17:09:00.502669846 +0000 UTC m=+39.111601197 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fb5065a0-5a41-40db-a798-5ccd320bdefe-samples-operator-tls") pod "cluster-samples-operator-6748489656-sbz9k" (UID: "fb5065a0-5a41-40db-a798-5ccd320bdefe") : secret "samples-operator-tls" not found Jul 28 17:08:58.503078 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:58.502737 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 17:08:58.503078 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:58.502768 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6ad5592-e8fc-41e7-8fcf-caa57c3eae35-cluster-monitoring-operator-tls podName:c6ad5592-e8fc-41e7-8fcf-caa57c3eae35 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:00.502758625 +0000 UTC m=+39.111689991 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c6ad5592-e8fc-41e7-8fcf-caa57c3eae35-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-m4bxm" (UID: "c6ad5592-e8fc-41e7-8fcf-caa57c3eae35") : secret "cluster-monitoring-operator-tls" not found Jul 28 17:08:58.503078 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:58.502833 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 28 17:08:58.503900 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:58.502864 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52730019-2aad-4136-91b5-1b82eb52e0d1-networking-console-plugin-cert podName:52730019-2aad-4136-91b5-1b82eb52e0d1 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:00.502853573 +0000 UTC m=+39.111784922 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/52730019-2aad-4136-91b5-1b82eb52e0d1-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-ngj99" (UID: "52730019-2aad-4136-91b5-1b82eb52e0d1") : secret "networking-console-plugin-cert" not found Jul 28 17:08:58.503900 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:58.502917 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:08:58.503900 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:58.502944 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd91b2e7-ff77-474f-a69b-47a84fe174a3-metrics-tls podName:fd91b2e7-ff77-474f-a69b-47a84fe174a3 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:00.502934393 +0000 UTC m=+39.111865757 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fd91b2e7-ff77-474f-a69b-47a84fe174a3-metrics-tls") pod "dns-default-49xwx" (UID: "fd91b2e7-ff77-474f-a69b-47a84fe174a3") : secret "dns-default-metrics-tls" not found Jul 28 17:08:58.503900 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:58.503008 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 17:08:58.503900 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:58.503018 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b5f57f94b-vgwb7: secret "image-registry-tls" not found Jul 28 17:08:58.503900 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:08:58.503046 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-registry-tls podName:2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c nodeName:}" failed. No retries permitted until 2026-07-28 17:09:00.503036646 +0000 UTC m=+39.111968008 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-registry-tls") pod "image-registry-6b5f57f94b-vgwb7" (UID: "2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c") : secret "image-registry-tls" not found Jul 28 17:08:59.453906 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:59.453864 2570 generic.go:356] "Generic (PLEG): container finished" podID="36c66dd3-6c06-4883-8ccd-107c566a45d6" containerID="0ce5ec9168d9e481cbbb921ec8c269e2e5dc7a95dc45d0971fe8ffad89dcac44" exitCode=0 Jul 28 17:08:59.454355 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:08:59.453926 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b9mxn" event={"ID":"36c66dd3-6c06-4883-8ccd-107c566a45d6","Type":"ContainerDied","Data":"0ce5ec9168d9e481cbbb921ec8c269e2e5dc7a95dc45d0971fe8ffad89dcac44"} Jul 28 17:09:00.462739 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:00.462455 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b9mxn" event={"ID":"36c66dd3-6c06-4883-8ccd-107c566a45d6","Type":"ContainerStarted","Data":"29f3d6e0d5ee3c52cef48a9f5ab84c40081d8aeec25e03518a6e8815a6fa23b2"} Jul 28 17:09:00.464659 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:00.464625 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-765pv" event={"ID":"ebeb51c6-5d43-493d-b8bd-e56f9993e032","Type":"ContainerStarted","Data":"4e4cbda1e2156e03da6aa86ddd52897443e9fc9656772676a793075617cfe020"} Jul 28 17:09:00.499582 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:00.499532 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-b9mxn" podStartSLOduration=9.645774697 podStartE2EDuration="38.499512565s" totalImagesPullingTime="28.853737868s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:08:22 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:00.497324945 +0000 UTC m=+39.106256315" watchObservedRunningTime="2026-07-28 17:09:00.499512565 +0000 UTC m=+39.108443917" Jul 28 17:09:00.521599 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:00.521546 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-765pv" podStartSLOduration=26.742341226 podStartE2EDuration="28.521524765s" totalImagesPullingTime="1.779183539s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:08:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:00.520582139 +0000 UTC m=+39.129513510" watchObservedRunningTime="2026-07-28 17:09:00.521524765 +0000 UTC m=+39.130456137" Jul 28 17:09:00.523991 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:00.523960 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb5065a0-5a41-40db-a798-5ccd320bdefe-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-sbz9k\" (UID: \"fb5065a0-5a41-40db-a798-5ccd320bdefe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-sbz9k" Jul 28 17:09:00.524161 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:00.524011 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c6ad5592-e8fc-41e7-8fcf-caa57c3eae35-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-m4bxm\" (UID: \"c6ad5592-e8fc-41e7-8fcf-caa57c3eae35\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-m4bxm" Jul 28 17:09:00.524161 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:00.524040 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/52730019-2aad-4136-91b5-1b82eb52e0d1-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-ngj99\" (UID: \"52730019-2aad-4136-91b5-1b82eb52e0d1\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-ngj99" Jul 28 17:09:00.524161 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:00.524073 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fd91b2e7-ff77-474f-a69b-47a84fe174a3-metrics-tls\") pod \"dns-default-49xwx\" (UID: \"fd91b2e7-ff77-474f-a69b-47a84fe174a3\") " pod="openshift-dns/dns-default-49xwx" Jul 28 17:09:00.524161 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:00.524133 2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 28 17:09:00.524388 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:00.524177 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-registry-tls\") pod \"image-registry-6b5f57f94b-vgwb7\" (UID: \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\") " pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" Jul 28 17:09:00.524388 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:00.524188 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb5065a0-5a41-40db-a798-5ccd320bdefe-samples-operator-tls podName:fb5065a0-5a41-40db-a798-5ccd320bdefe nodeName:}" failed. No retries permitted until 2026-07-28 17:09:04.524172984 +0000 UTC m=+43.133104355 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fb5065a0-5a41-40db-a798-5ccd320bdefe-samples-operator-tls") pod "cluster-samples-operator-6748489656-sbz9k" (UID: "fb5065a0-5a41-40db-a798-5ccd320bdefe") : secret "samples-operator-tls" not found Jul 28 17:09:00.524388 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:00.524232 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af095ddd-e1cd-4148-9796-d60be4fef0e2-metrics-certs\") pod \"router-default-64c477cb5c-9stkr\" (UID: \"af095ddd-e1cd-4148-9796-d60be4fef0e2\") " pod="openshift-ingress/router-default-64c477cb5c-9stkr" Jul 28 17:09:00.524388 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:00.524260 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 17:09:00.524388 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:00.524271 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b5f57f94b-vgwb7: secret "image-registry-tls" not found Jul 28 17:09:00.524388 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:00.524305 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-registry-tls podName:2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c nodeName:}" failed. No retries permitted until 2026-07-28 17:09:04.524294826 +0000 UTC m=+43.133226191 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-registry-tls") pod "image-registry-6b5f57f94b-vgwb7" (UID: "2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c") : secret "image-registry-tls" not found Jul 28 17:09:00.524388 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:00.524268 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af095ddd-e1cd-4148-9796-d60be4fef0e2-service-ca-bundle\") pod \"router-default-64c477cb5c-9stkr\" (UID: \"af095ddd-e1cd-4148-9796-d60be4fef0e2\") " pod="openshift-ingress/router-default-64c477cb5c-9stkr" Jul 28 17:09:00.524388 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:00.524330 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/af095ddd-e1cd-4148-9796-d60be4fef0e2-service-ca-bundle podName:af095ddd-e1cd-4148-9796-d60be4fef0e2 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:04.52432067 +0000 UTC m=+43.133252018 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/af095ddd-e1cd-4148-9796-d60be4fef0e2-service-ca-bundle") pod "router-default-64c477cb5c-9stkr" (UID: "af095ddd-e1cd-4148-9796-d60be4fef0e2") : configmap references non-existent config key: service-ca.crt Jul 28 17:09:00.524388 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:00.524367 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 17:09:00.524836 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:00.524397 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6ad5592-e8fc-41e7-8fcf-caa57c3eae35-cluster-monitoring-operator-tls podName:c6ad5592-e8fc-41e7-8fcf-caa57c3eae35 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:04.52438895 +0000 UTC m=+43.133320314 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c6ad5592-e8fc-41e7-8fcf-caa57c3eae35-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-m4bxm" (UID: "c6ad5592-e8fc-41e7-8fcf-caa57c3eae35") : secret "cluster-monitoring-operator-tls" not found Jul 28 17:09:00.524836 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:00.524446 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 28 17:09:00.524836 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:00.524475 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52730019-2aad-4136-91b5-1b82eb52e0d1-networking-console-plugin-cert podName:52730019-2aad-4136-91b5-1b82eb52e0d1 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:04.524465645 +0000 UTC m=+43.133396999 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/52730019-2aad-4136-91b5-1b82eb52e0d1-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-ngj99" (UID: "52730019-2aad-4136-91b5-1b82eb52e0d1") : secret "networking-console-plugin-cert" not found Jul 28 17:09:00.524836 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:00.524522 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:09:00.524836 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:00.524549 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd91b2e7-ff77-474f-a69b-47a84fe174a3-metrics-tls podName:fd91b2e7-ff77-474f-a69b-47a84fe174a3 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:04.524540416 +0000 UTC m=+43.133471765 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fd91b2e7-ff77-474f-a69b-47a84fe174a3-metrics-tls") pod "dns-default-49xwx" (UID: "fd91b2e7-ff77-474f-a69b-47a84fe174a3") : secret "dns-default-metrics-tls" not found Jul 28 17:09:00.524836 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:00.524596 2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 28 17:09:00.524836 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:00.524623 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af095ddd-e1cd-4148-9796-d60be4fef0e2-metrics-certs podName:af095ddd-e1cd-4148-9796-d60be4fef0e2 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:04.524614578 +0000 UTC m=+43.133545929 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af095ddd-e1cd-4148-9796-d60be4fef0e2-metrics-certs") pod "router-default-64c477cb5c-9stkr" (UID: "af095ddd-e1cd-4148-9796-d60be4fef0e2") : secret "router-metrics-certs-default" not found Jul 28 17:09:04.560154 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:04.560098 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c6ad5592-e8fc-41e7-8fcf-caa57c3eae35-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-m4bxm\" (UID: \"c6ad5592-e8fc-41e7-8fcf-caa57c3eae35\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-m4bxm" Jul 28 17:09:04.560919 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:04.560172 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/52730019-2aad-4136-91b5-1b82eb52e0d1-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-ngj99\" (UID: \"52730019-2aad-4136-91b5-1b82eb52e0d1\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-ngj99" Jul 28 17:09:04.560919 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:04.560216 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fd91b2e7-ff77-474f-a69b-47a84fe174a3-metrics-tls\") pod \"dns-default-49xwx\" (UID: \"fd91b2e7-ff77-474f-a69b-47a84fe174a3\") " pod="openshift-dns/dns-default-49xwx" Jul 28 17:09:04.560919 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:04.560254 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 17:09:04.560919 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:04.560292 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-registry-tls\") pod \"image-registry-6b5f57f94b-vgwb7\" (UID: \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\") " pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" Jul 28 17:09:04.560919 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:04.560314 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af095ddd-e1cd-4148-9796-d60be4fef0e2-metrics-certs\") pod \"router-default-64c477cb5c-9stkr\" (UID: \"af095ddd-e1cd-4148-9796-d60be4fef0e2\") " pod="openshift-ingress/router-default-64c477cb5c-9stkr" Jul 28 17:09:04.560919 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:04.560348 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6ad5592-e8fc-41e7-8fcf-caa57c3eae35-cluster-monitoring-operator-tls podName:c6ad5592-e8fc-41e7-8fcf-caa57c3eae35 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:12.560325883 +0000 UTC m=+51.169257237 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c6ad5592-e8fc-41e7-8fcf-caa57c3eae35-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-m4bxm" (UID: "c6ad5592-e8fc-41e7-8fcf-caa57c3eae35") : secret "cluster-monitoring-operator-tls" not found Jul 28 17:09:04.560919 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:04.560354 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 28 17:09:04.560919 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:04.560381 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af095ddd-e1cd-4148-9796-d60be4fef0e2-service-ca-bundle\") pod \"router-default-64c477cb5c-9stkr\" (UID: \"af095ddd-e1cd-4148-9796-d60be4fef0e2\") " pod="openshift-ingress/router-default-64c477cb5c-9stkr" Jul 28 17:09:04.560919 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:04.560392 2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 28 17:09:04.560919 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:04.560417 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 17:09:04.560919 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:04.560428 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52730019-2aad-4136-91b5-1b82eb52e0d1-networking-console-plugin-cert podName:52730019-2aad-4136-91b5-1b82eb52e0d1 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:12.560410601 +0000 UTC m=+51.169341962 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/52730019-2aad-4136-91b5-1b82eb52e0d1-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-ngj99" (UID: "52730019-2aad-4136-91b5-1b82eb52e0d1") : secret "networking-console-plugin-cert" not found Jul 28 17:09:04.560919 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:04.560428 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:09:04.560919 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:04.560445 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b5f57f94b-vgwb7: secret "image-registry-tls" not found Jul 28 17:09:04.560919 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:04.560470 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/af095ddd-e1cd-4148-9796-d60be4fef0e2-service-ca-bundle podName:af095ddd-e1cd-4148-9796-d60be4fef0e2 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:12.560459654 +0000 UTC m=+51.169391001 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/af095ddd-e1cd-4148-9796-d60be4fef0e2-service-ca-bundle") pod "router-default-64c477cb5c-9stkr" (UID: "af095ddd-e1cd-4148-9796-d60be4fef0e2") : configmap references non-existent config key: service-ca.crt Jul 28 17:09:04.560919 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:04.560470 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb5065a0-5a41-40db-a798-5ccd320bdefe-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-sbz9k\" (UID: \"fb5065a0-5a41-40db-a798-5ccd320bdefe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-sbz9k" Jul 28 17:09:04.560919 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:04.560492 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd91b2e7-ff77-474f-a69b-47a84fe174a3-metrics-tls podName:fd91b2e7-ff77-474f-a69b-47a84fe174a3 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:12.560477155 +0000 UTC m=+51.169408502 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fd91b2e7-ff77-474f-a69b-47a84fe174a3-metrics-tls") pod "dns-default-49xwx" (UID: "fd91b2e7-ff77-474f-a69b-47a84fe174a3") : secret "dns-default-metrics-tls" not found Jul 28 17:09:04.561535 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:04.560514 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-registry-tls podName:2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c nodeName:}" failed. No retries permitted until 2026-07-28 17:09:12.560505865 +0000 UTC m=+51.169437214 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-registry-tls") pod "image-registry-6b5f57f94b-vgwb7" (UID: "2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c") : secret "image-registry-tls" not found Jul 28 17:09:04.561535 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:04.560529 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af095ddd-e1cd-4148-9796-d60be4fef0e2-metrics-certs podName:af095ddd-e1cd-4148-9796-d60be4fef0e2 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:12.560521825 +0000 UTC m=+51.169453172 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af095ddd-e1cd-4148-9796-d60be4fef0e2-metrics-certs") pod "router-default-64c477cb5c-9stkr" (UID: "af095ddd-e1cd-4148-9796-d60be4fef0e2") : secret "router-metrics-certs-default" not found Jul 28 17:09:04.561535 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:04.560535 2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 28 17:09:04.561535 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:04.560571 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb5065a0-5a41-40db-a798-5ccd320bdefe-samples-operator-tls podName:fb5065a0-5a41-40db-a798-5ccd320bdefe nodeName:}" failed. No retries permitted until 2026-07-28 17:09:12.560559211 +0000 UTC m=+51.169490585 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fb5065a0-5a41-40db-a798-5ccd320bdefe-samples-operator-tls") pod "cluster-samples-operator-6748489656-sbz9k" (UID: "fb5065a0-5a41-40db-a798-5ccd320bdefe") : secret "samples-operator-tls" not found Jul 28 17:09:05.480725 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:05.480644 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-bpkxs" event={"ID":"8f9b8ad8-774d-4ab8-9826-94eb5a37743f","Type":"ContainerStarted","Data":"b1a154d20b8f9888cf5dc3364314d24277362efbc6a5c3b400b41d002faa49ce"} Jul 28 17:09:05.483241 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:05.483218 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-slfqf_ecb82800-8cf0-4ef9-926c-155172fad7ff/console-operator/0.log" Jul 28 17:09:05.483374 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:05.483257 2570 generic.go:356] "Generic (PLEG): container finished" podID="ecb82800-8cf0-4ef9-926c-155172fad7ff" containerID="537afd1a219d829c2d54b8e6336fd24d1da6e402efcf546d46ca352ca0137483" exitCode=255 Jul 28 17:09:05.483374 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:05.483336 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-slfqf" event={"ID":"ecb82800-8cf0-4ef9-926c-155172fad7ff","Type":"ContainerDied","Data":"537afd1a219d829c2d54b8e6336fd24d1da6e402efcf546d46ca352ca0137483"} Jul 28 17:09:05.483620 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:05.483603 2570 scope.go:117] "RemoveContainer" containerID="537afd1a219d829c2d54b8e6336fd24d1da6e402efcf546d46ca352ca0137483" Jul 28 17:09:05.488179 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:05.488152 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-xx7xr" event={"ID":"551cb4c6-f787-4677-b62a-c658a20782d6","Type":"ContainerStarted","Data":"fa20fdd935ac7a49c12fbab8b9d20b0b3999e21864ccd33630da5594cb692871"} Jul 28 17:09:05.490841 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:05.490456 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-tl7rc" event={"ID":"9a22bd3d-e2c0-461e-b364-b02d99e87eb9","Type":"ContainerStarted","Data":"10e403ba03868c5ac447e7fc342b40fbade6d4009ed6598050f243c8fc7053c9"} Jul 28 17:09:05.492791 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:05.492256 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-s9t99" event={"ID":"78838e40-d32b-4f26-9e51-3dd3352026c3","Type":"ContainerStarted","Data":"a085dceb8aadb411122bee4159d8987afee88339e7754823b54a808a2e73c891"} Jul 28 17:09:05.495276 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:05.494926 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-vznd7" event={"ID":"4ddc4d25-529e-44f1-a922-1516040c72f4","Type":"ContainerStarted","Data":"5085a92bdd988494ceba8591e52367cf0a8f7d262b5d08b1d9a95f6f5c5c5a57"} Jul 28 17:09:05.502827 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:05.502769 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-bpkxs" podStartSLOduration=26.077127609 podStartE2EDuration="33.502753041s" totalImagesPullingTime="7.425625432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:08:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:05.501843369 +0000 UTC m=+44.110774739" watchObservedRunningTime="2026-07-28 17:09:05.502753041 +0000 UTC m=+44.111684411" Jul 28 17:09:05.546905 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:05.546697 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-vznd7" podStartSLOduration=26.124737555 podStartE2EDuration="33.546676355s" totalImagesPullingTime="7.4219388s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:08:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:05.522790758 +0000 UTC m=+44.131722130" watchObservedRunningTime="2026-07-28 17:09:05.546676355 +0000 UTC m=+44.155607727" Jul 28 17:09:05.580737 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:05.572519 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-tl7rc" podStartSLOduration=26.108717135 podStartE2EDuration="33.57249268s" totalImagesPullingTime="7.463775545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:08:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:05.569083738 +0000 UTC m=+44.178015110" watchObservedRunningTime="2026-07-28 17:09:05.57249268 +0000 UTC m=+44.181424052" Jul 28 17:09:05.598653 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:05.598592 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-xx7xr" podStartSLOduration=26.16550167 podStartE2EDuration="33.598571129s" totalImagesPullingTime="7.433069459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:08:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:05.597438056 +0000 UTC m=+44.206369427" watchObservedRunningTime="2026-07-28 17:09:05.598571129 +0000 UTC m=+44.207502502" Jul 28 17:09:05.618754 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:05.618637 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-s9t99" podStartSLOduration=2.194302065 podStartE2EDuration="9.618613835s" totalImagesPullingTime="7.42431177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:08:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:05.617013768 +0000 UTC m=+44.225945139" watchObservedRunningTime="2026-07-28 17:09:05.618613835 +0000 UTC m=+44.227545206" Jul 28 17:09:06.353661 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:06.353623 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-n776k"] Jul 28 17:09:06.376036 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:06.376001 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-n776k"] Jul 28 17:09:06.376197 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:06.376156 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-n776k" Jul 28 17:09:06.380422 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:06.380399 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 28 17:09:06.380641 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:06.380434 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 28 17:09:06.380709 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:06.380498 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-k4zqq\"" Jul 28 17:09:06.490127 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:06.490030 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/39aa1164-b895-45e9-b64a-5220376fdd95-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n776k\" (UID: \"39aa1164-b895-45e9-b64a-5220376fdd95\") " pod="openshift-insights/insights-runtime-extractor-n776k" Jul 28 17:09:06.490127 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:06.490079 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/39aa1164-b895-45e9-b64a-5220376fdd95-crio-socket\") pod \"insights-runtime-extractor-n776k\" (UID: \"39aa1164-b895-45e9-b64a-5220376fdd95\") " pod="openshift-insights/insights-runtime-extractor-n776k" Jul 28 17:09:06.490318 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:06.490264 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sxtr\" (UniqueName: \"kubernetes.io/projected/39aa1164-b895-45e9-b64a-5220376fdd95-kube-api-access-7sxtr\") pod \"insights-runtime-extractor-n776k\" (UID: \"39aa1164-b895-45e9-b64a-5220376fdd95\") " pod="openshift-insights/insights-runtime-extractor-n776k" Jul 28 17:09:06.490354 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:06.490325 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/39aa1164-b895-45e9-b64a-5220376fdd95-data-volume\") pod \"insights-runtime-extractor-n776k\" (UID: \"39aa1164-b895-45e9-b64a-5220376fdd95\") " pod="openshift-insights/insights-runtime-extractor-n776k" Jul 28 17:09:06.490387 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:06.490370 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/39aa1164-b895-45e9-b64a-5220376fdd95-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n776k\" (UID: \"39aa1164-b895-45e9-b64a-5220376fdd95\") " pod="openshift-insights/insights-runtime-extractor-n776k" Jul 28 17:09:06.501009 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:06.500982 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-slfqf_ecb82800-8cf0-4ef9-926c-155172fad7ff/console-operator/1.log" Jul 28 17:09:06.501503 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:06.501484 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-slfqf_ecb82800-8cf0-4ef9-926c-155172fad7ff/console-operator/0.log" Jul 28 17:09:06.501567 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:06.501528 2570 generic.go:356] "Generic (PLEG): container finished" podID="ecb82800-8cf0-4ef9-926c-155172fad7ff" containerID="f498d6b266d4f167d89591a8deccfc0c8021dd46231fa63d5e33d193c60ba99c" exitCode=255 Jul 28 17:09:06.501649 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:06.501622 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-slfqf" event={"ID":"ecb82800-8cf0-4ef9-926c-155172fad7ff","Type":"ContainerDied","Data":"f498d6b266d4f167d89591a8deccfc0c8021dd46231fa63d5e33d193c60ba99c"} Jul 28 17:09:06.501704 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:06.501686 2570 scope.go:117] "RemoveContainer" containerID="537afd1a219d829c2d54b8e6336fd24d1da6e402efcf546d46ca352ca0137483" Jul 28 17:09:06.501917 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:06.501900 2570 scope.go:117] "RemoveContainer" containerID="f498d6b266d4f167d89591a8deccfc0c8021dd46231fa63d5e33d193c60ba99c" Jul 28 17:09:06.502104 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:06.502082 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-slfqf_openshift-console-operator(ecb82800-8cf0-4ef9-926c-155172fad7ff)\"" pod="openshift-console-operator/console-operator-584dcb6b54-slfqf" podUID="ecb82800-8cf0-4ef9-926c-155172fad7ff" Jul 28 17:09:06.591659 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:06.591617 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sxtr\" (UniqueName: \"kubernetes.io/projected/39aa1164-b895-45e9-b64a-5220376fdd95-kube-api-access-7sxtr\") pod \"insights-runtime-extractor-n776k\" (UID: \"39aa1164-b895-45e9-b64a-5220376fdd95\") " pod="openshift-insights/insights-runtime-extractor-n776k" Jul 28 17:09:06.592187 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:06.591699 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/39aa1164-b895-45e9-b64a-5220376fdd95-data-volume\") pod \"insights-runtime-extractor-n776k\" (UID: \"39aa1164-b895-45e9-b64a-5220376fdd95\") " pod="openshift-insights/insights-runtime-extractor-n776k" Jul 28 17:09:06.592187 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:06.591752 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/39aa1164-b895-45e9-b64a-5220376fdd95-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n776k\" (UID: \"39aa1164-b895-45e9-b64a-5220376fdd95\") " pod="openshift-insights/insights-runtime-extractor-n776k" Jul 28 17:09:06.592187 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:06.591911 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/39aa1164-b895-45e9-b64a-5220376fdd95-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n776k\" (UID: \"39aa1164-b895-45e9-b64a-5220376fdd95\") " pod="openshift-insights/insights-runtime-extractor-n776k" Jul 28 17:09:06.592187 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:06.591968 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/39aa1164-b895-45e9-b64a-5220376fdd95-crio-socket\") pod \"insights-runtime-extractor-n776k\" (UID: \"39aa1164-b895-45e9-b64a-5220376fdd95\") " pod="openshift-insights/insights-runtime-extractor-n776k" Jul 28 17:09:06.592187 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:06.592082 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/39aa1164-b895-45e9-b64a-5220376fdd95-data-volume\") pod \"insights-runtime-extractor-n776k\" (UID: \"39aa1164-b895-45e9-b64a-5220376fdd95\") " pod="openshift-insights/insights-runtime-extractor-n776k" Jul 28 17:09:06.592490 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:06.591976 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 17:09:06.592490 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:06.592264 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/39aa1164-b895-45e9-b64a-5220376fdd95-crio-socket\") pod \"insights-runtime-extractor-n776k\" (UID: \"39aa1164-b895-45e9-b64a-5220376fdd95\") " pod="openshift-insights/insights-runtime-extractor-n776k" Jul 28 17:09:06.592490 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:06.592277 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/39aa1164-b895-45e9-b64a-5220376fdd95-kube-rbac-proxy-cm podName:39aa1164-b895-45e9-b64a-5220376fdd95 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:07.092254388 +0000 UTC m=+45.701185747 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/39aa1164-b895-45e9-b64a-5220376fdd95-kube-rbac-proxy-cm") pod "insights-runtime-extractor-n776k" (UID: "39aa1164-b895-45e9-b64a-5220376fdd95") : configmap "kube-rbac-proxy" not found Jul 28 17:09:06.592670 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:06.592520 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 17:09:06.592670 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:06.592564 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39aa1164-b895-45e9-b64a-5220376fdd95-insights-runtime-extractor-tls podName:39aa1164-b895-45e9-b64a-5220376fdd95 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:07.092549554 +0000 UTC m=+45.701480909 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/39aa1164-b895-45e9-b64a-5220376fdd95-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n776k" (UID: "39aa1164-b895-45e9-b64a-5220376fdd95") : secret "insights-runtime-extractor-tls" not found Jul 28 17:09:06.604716 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:06.604687 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sxtr\" (UniqueName: \"kubernetes.io/projected/39aa1164-b895-45e9-b64a-5220376fdd95-kube-api-access-7sxtr\") pod \"insights-runtime-extractor-n776k\" (UID: \"39aa1164-b895-45e9-b64a-5220376fdd95\") " pod="openshift-insights/insights-runtime-extractor-n776k" Jul 28 17:09:07.031302 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:07.031261 2570 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-slfqf" Jul 28 17:09:07.031473 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:07.031316 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-slfqf" Jul 28 17:09:07.097132 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:07.097080 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/39aa1164-b895-45e9-b64a-5220376fdd95-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n776k\" (UID: \"39aa1164-b895-45e9-b64a-5220376fdd95\") " pod="openshift-insights/insights-runtime-extractor-n776k" Jul 28 17:09:07.097308 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:07.097179 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/39aa1164-b895-45e9-b64a-5220376fdd95-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n776k\" (UID: \"39aa1164-b895-45e9-b64a-5220376fdd95\") " pod="openshift-insights/insights-runtime-extractor-n776k" Jul 28 17:09:07.097308 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:07.097230 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 17:09:07.097308 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:07.097274 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 17:09:07.097308 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:07.097299 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39aa1164-b895-45e9-b64a-5220376fdd95-insights-runtime-extractor-tls podName:39aa1164-b895-45e9-b64a-5220376fdd95 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:08.097282585 +0000 UTC m=+46.706213933 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/39aa1164-b895-45e9-b64a-5220376fdd95-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n776k" (UID: "39aa1164-b895-45e9-b64a-5220376fdd95") : secret "insights-runtime-extractor-tls" not found Jul 28 17:09:07.097507 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:07.097316 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/39aa1164-b895-45e9-b64a-5220376fdd95-kube-rbac-proxy-cm podName:39aa1164-b895-45e9-b64a-5220376fdd95 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:08.097304367 +0000 UTC m=+46.706235715 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/39aa1164-b895-45e9-b64a-5220376fdd95-kube-rbac-proxy-cm") pod "insights-runtime-extractor-n776k" (UID: "39aa1164-b895-45e9-b64a-5220376fdd95") : configmap "kube-rbac-proxy" not found Jul 28 17:09:07.506177 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:07.506148 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-slfqf_ecb82800-8cf0-4ef9-926c-155172fad7ff/console-operator/1.log" Jul 28 17:09:07.506498 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:07.506482 2570 scope.go:117] "RemoveContainer" containerID="f498d6b266d4f167d89591a8deccfc0c8021dd46231fa63d5e33d193c60ba99c" Jul 28 17:09:07.506649 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:07.506631 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-slfqf_openshift-console-operator(ecb82800-8cf0-4ef9-926c-155172fad7ff)\"" pod="openshift-console-operator/console-operator-584dcb6b54-slfqf" podUID="ecb82800-8cf0-4ef9-926c-155172fad7ff" Jul 28 17:09:08.107557 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:08.107516 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/39aa1164-b895-45e9-b64a-5220376fdd95-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n776k\" (UID: \"39aa1164-b895-45e9-b64a-5220376fdd95\") " pod="openshift-insights/insights-runtime-extractor-n776k" Jul 28 17:09:08.107907 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:08.107601 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/39aa1164-b895-45e9-b64a-5220376fdd95-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n776k\" (UID: \"39aa1164-b895-45e9-b64a-5220376fdd95\") " pod="openshift-insights/insights-runtime-extractor-n776k" Jul 28 17:09:08.107907 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:08.107665 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 17:09:08.107907 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:08.107689 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 17:09:08.107907 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:08.107759 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/39aa1164-b895-45e9-b64a-5220376fdd95-kube-rbac-proxy-cm podName:39aa1164-b895-45e9-b64a-5220376fdd95 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:10.107727847 +0000 UTC m=+48.716659199 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/39aa1164-b895-45e9-b64a-5220376fdd95-kube-rbac-proxy-cm") pod "insights-runtime-extractor-n776k" (UID: "39aa1164-b895-45e9-b64a-5220376fdd95") : configmap "kube-rbac-proxy" not found Jul 28 17:09:08.107907 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:08.107779 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39aa1164-b895-45e9-b64a-5220376fdd95-insights-runtime-extractor-tls podName:39aa1164-b895-45e9-b64a-5220376fdd95 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:10.107768648 +0000 UTC m=+48.716700002 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/39aa1164-b895-45e9-b64a-5220376fdd95-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n776k" (UID: "39aa1164-b895-45e9-b64a-5220376fdd95") : secret "insights-runtime-extractor-tls" not found Jul 28 17:09:08.135452 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:08.135427 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zbmdl_dd54c89e-8c35-404b-8804-ca3e45d384b4/dns-node-resolver/0.log" Jul 28 17:09:08.509830 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:08.509804 2570 scope.go:117] "RemoveContainer" containerID="f498d6b266d4f167d89591a8deccfc0c8021dd46231fa63d5e33d193c60ba99c" Jul 28 17:09:08.509989 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:08.509963 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-slfqf_openshift-console-operator(ecb82800-8cf0-4ef9-926c-155172fad7ff)\"" pod="openshift-console-operator/console-operator-584dcb6b54-slfqf" podUID="ecb82800-8cf0-4ef9-926c-155172fad7ff" Jul 28 17:09:08.933254 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:08.933226 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cmx6g_56d1a325-976b-4060-b1a4-1a4b6be0bd18/node-ca/0.log" Jul 28 17:09:09.528944 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:09.528917 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-s9t99_78838e40-d32b-4f26-9e51-3dd3352026c3/serve-healthcheck-canary/0.log" Jul 28 17:09:10.128204 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:10.128175 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/39aa1164-b895-45e9-b64a-5220376fdd95-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n776k\" (UID: \"39aa1164-b895-45e9-b64a-5220376fdd95\") " pod="openshift-insights/insights-runtime-extractor-n776k" Jul 28 17:09:10.128385 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:10.128316 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 17:09:10.128385 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:10.128337 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/39aa1164-b895-45e9-b64a-5220376fdd95-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n776k\" (UID: \"39aa1164-b895-45e9-b64a-5220376fdd95\") " pod="openshift-insights/insights-runtime-extractor-n776k" Jul 28 17:09:10.128385 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:10.128367 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39aa1164-b895-45e9-b64a-5220376fdd95-insights-runtime-extractor-tls podName:39aa1164-b895-45e9-b64a-5220376fdd95 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:14.12835293 +0000 UTC m=+52.737284278 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/39aa1164-b895-45e9-b64a-5220376fdd95-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n776k" (UID: "39aa1164-b895-45e9-b64a-5220376fdd95") : secret "insights-runtime-extractor-tls" not found Jul 28 17:09:10.128518 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:10.128388 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 17:09:10.128518 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:10.128435 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/39aa1164-b895-45e9-b64a-5220376fdd95-kube-rbac-proxy-cm podName:39aa1164-b895-45e9-b64a-5220376fdd95 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:14.128423756 +0000 UTC m=+52.737355103 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/39aa1164-b895-45e9-b64a-5220376fdd95-kube-rbac-proxy-cm") pod "insights-runtime-extractor-n776k" (UID: "39aa1164-b895-45e9-b64a-5220376fdd95") : configmap "kube-rbac-proxy" not found Jul 28 17:09:12.648450 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:12.648413 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb5065a0-5a41-40db-a798-5ccd320bdefe-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-sbz9k\" (UID: \"fb5065a0-5a41-40db-a798-5ccd320bdefe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-sbz9k" Jul 28 17:09:12.648450 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:12.648462 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c6ad5592-e8fc-41e7-8fcf-caa57c3eae35-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-m4bxm\" (UID: \"c6ad5592-e8fc-41e7-8fcf-caa57c3eae35\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-m4bxm" Jul 28 17:09:12.648925 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:12.648484 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/52730019-2aad-4136-91b5-1b82eb52e0d1-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-ngj99\" (UID: \"52730019-2aad-4136-91b5-1b82eb52e0d1\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-ngj99" Jul 28 17:09:12.648925 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:12.648519 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fd91b2e7-ff77-474f-a69b-47a84fe174a3-metrics-tls\") pod \"dns-default-49xwx\" (UID: \"fd91b2e7-ff77-474f-a69b-47a84fe174a3\") " pod="openshift-dns/dns-default-49xwx" Jul 28 17:09:12.648925 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:12.648570 2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 28 17:09:12.648925 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:12.648571 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-registry-tls\") pod \"image-registry-6b5f57f94b-vgwb7\" (UID: \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\") " pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" Jul 28 17:09:12.648925 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:12.648570 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 17:09:12.648925 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:12.648638 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb5065a0-5a41-40db-a798-5ccd320bdefe-samples-operator-tls podName:fb5065a0-5a41-40db-a798-5ccd320bdefe nodeName:}" failed. No retries permitted until 2026-07-28 17:09:28.648618641 +0000 UTC m=+67.257550008 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fb5065a0-5a41-40db-a798-5ccd320bdefe-samples-operator-tls") pod "cluster-samples-operator-6748489656-sbz9k" (UID: "fb5065a0-5a41-40db-a798-5ccd320bdefe") : secret "samples-operator-tls" not found Jul 28 17:09:12.648925 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:12.648653 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 17:09:12.648925 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:12.648666 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b5f57f94b-vgwb7: secret "image-registry-tls" not found Jul 28 17:09:12.648925 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:12.648669 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 28 17:09:12.648925 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:12.648681 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af095ddd-e1cd-4148-9796-d60be4fef0e2-metrics-certs\") pod \"router-default-64c477cb5c-9stkr\" (UID: \"af095ddd-e1cd-4148-9796-d60be4fef0e2\") " pod="openshift-ingress/router-default-64c477cb5c-9stkr" Jul 28 17:09:12.648925 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:12.648693 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6ad5592-e8fc-41e7-8fcf-caa57c3eae35-cluster-monitoring-operator-tls podName:c6ad5592-e8fc-41e7-8fcf-caa57c3eae35 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:28.648674863 +0000 UTC m=+67.257606215 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c6ad5592-e8fc-41e7-8fcf-caa57c3eae35-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-m4bxm" (UID: "c6ad5592-e8fc-41e7-8fcf-caa57c3eae35") : secret "cluster-monitoring-operator-tls" not found Jul 28 17:09:12.648925 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:12.648618 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:09:12.648925 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:12.648721 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-registry-tls podName:2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c nodeName:}" failed. No retries permitted until 2026-07-28 17:09:28.64870643 +0000 UTC m=+67.257637778 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-registry-tls") pod "image-registry-6b5f57f94b-vgwb7" (UID: "2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c") : secret "image-registry-tls" not found Jul 28 17:09:12.648925 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:12.648741 2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 28 17:09:12.648925 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:12.648747 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52730019-2aad-4136-91b5-1b82eb52e0d1-networking-console-plugin-cert podName:52730019-2aad-4136-91b5-1b82eb52e0d1 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:28.648740404 +0000 UTC m=+67.257671752 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/52730019-2aad-4136-91b5-1b82eb52e0d1-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-ngj99" (UID: "52730019-2aad-4136-91b5-1b82eb52e0d1") : secret "networking-console-plugin-cert" not found Jul 28 17:09:12.648925 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:12.648758 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd91b2e7-ff77-474f-a69b-47a84fe174a3-metrics-tls podName:fd91b2e7-ff77-474f-a69b-47a84fe174a3 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:28.648753226 +0000 UTC m=+67.257684574 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fd91b2e7-ff77-474f-a69b-47a84fe174a3-metrics-tls") pod "dns-default-49xwx" (UID: "fd91b2e7-ff77-474f-a69b-47a84fe174a3") : secret "dns-default-metrics-tls" not found Jul 28 17:09:12.649514 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:12.648772 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af095ddd-e1cd-4148-9796-d60be4fef0e2-metrics-certs podName:af095ddd-e1cd-4148-9796-d60be4fef0e2 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:28.648764935 +0000 UTC m=+67.257696282 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af095ddd-e1cd-4148-9796-d60be4fef0e2-metrics-certs") pod "router-default-64c477cb5c-9stkr" (UID: "af095ddd-e1cd-4148-9796-d60be4fef0e2") : secret "router-metrics-certs-default" not found Jul 28 17:09:12.649514 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:12.648800 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af095ddd-e1cd-4148-9796-d60be4fef0e2-service-ca-bundle\") pod \"router-default-64c477cb5c-9stkr\" (UID: \"af095ddd-e1cd-4148-9796-d60be4fef0e2\") " pod="openshift-ingress/router-default-64c477cb5c-9stkr" Jul 28 17:09:12.649514 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:12.648875 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/af095ddd-e1cd-4148-9796-d60be4fef0e2-service-ca-bundle podName:af095ddd-e1cd-4148-9796-d60be4fef0e2 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:28.648869021 +0000 UTC m=+67.257800369 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/af095ddd-e1cd-4148-9796-d60be4fef0e2-service-ca-bundle") pod "router-default-64c477cb5c-9stkr" (UID: "af095ddd-e1cd-4148-9796-d60be4fef0e2") : configmap references non-existent config key: service-ca.crt Jul 28 17:09:14.162492 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:14.162452 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/39aa1164-b895-45e9-b64a-5220376fdd95-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n776k\" (UID: \"39aa1164-b895-45e9-b64a-5220376fdd95\") " pod="openshift-insights/insights-runtime-extractor-n776k" Jul 28 17:09:14.162880 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:14.162613 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/39aa1164-b895-45e9-b64a-5220376fdd95-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n776k\" (UID: \"39aa1164-b895-45e9-b64a-5220376fdd95\") " pod="openshift-insights/insights-runtime-extractor-n776k" Jul 28 17:09:14.162880 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:14.162619 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 17:09:14.162880 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:09:14.162688 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39aa1164-b895-45e9-b64a-5220376fdd95-insights-runtime-extractor-tls podName:39aa1164-b895-45e9-b64a-5220376fdd95 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:22.162666865 +0000 UTC m=+60.771598214 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/39aa1164-b895-45e9-b64a-5220376fdd95-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n776k" (UID: "39aa1164-b895-45e9-b64a-5220376fdd95") : secret "insights-runtime-extractor-tls" not found Jul 28 17:09:14.163214 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:14.163199 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/39aa1164-b895-45e9-b64a-5220376fdd95-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n776k\" (UID: \"39aa1164-b895-45e9-b64a-5220376fdd95\") " pod="openshift-insights/insights-runtime-extractor-n776k" Jul 28 17:09:20.113839 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:20.113804 2570 scope.go:117] "RemoveContainer" containerID="f498d6b266d4f167d89591a8deccfc0c8021dd46231fa63d5e33d193c60ba99c" Jul 28 17:09:20.548863 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:20.548836 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-slfqf_ecb82800-8cf0-4ef9-926c-155172fad7ff/console-operator/1.log" Jul 28 17:09:20.549018 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:20.548918 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-slfqf" event={"ID":"ecb82800-8cf0-4ef9-926c-155172fad7ff","Type":"ContainerStarted","Data":"66230f04c0ccc0807d10e1b647d212afa8500c7b99b5231e2709796d4a392c6c"} Jul 28 17:09:20.549174 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:20.549150 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-slfqf" Jul 28 17:09:20.555975 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:20.555941 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-slfqf" Jul 28 17:09:20.585696 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:20.585636 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-slfqf" podStartSLOduration=41.161819786 podStartE2EDuration="48.585621873s" totalImagesPullingTime="7.423802087s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:08:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:20.584074224 +0000 UTC m=+59.193005613" watchObservedRunningTime="2026-07-28 17:09:20.585621873 +0000 UTC m=+59.194553287" Jul 28 17:09:22.234469 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:22.234425 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/39aa1164-b895-45e9-b64a-5220376fdd95-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n776k\" (UID: \"39aa1164-b895-45e9-b64a-5220376fdd95\") " pod="openshift-insights/insights-runtime-extractor-n776k" Jul 28 17:09:22.236909 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:22.236881 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/39aa1164-b895-45e9-b64a-5220376fdd95-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n776k\" (UID: \"39aa1164-b895-45e9-b64a-5220376fdd95\") " pod="openshift-insights/insights-runtime-extractor-n776k" Jul 28 17:09:22.290750 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:22.290717 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-k4zqq\"" Jul 28 17:09:22.298233 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:22.298212 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-n776k" Jul 28 17:09:22.394610 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:22.394532 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x6gbq" Jul 28 17:09:22.466090 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:22.466064 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-n776k"] Jul 28 17:09:22.468873 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:09:22.468851 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39aa1164_b895_45e9_b64a_5220376fdd95.slice/crio-f7a22313a50a709ced0777e3086cc05a4fb7bc10554659cb437a689019be20e4 WatchSource:0}: Error finding container f7a22313a50a709ced0777e3086cc05a4fb7bc10554659cb437a689019be20e4: Status 404 returned error can't find the container with id f7a22313a50a709ced0777e3086cc05a4fb7bc10554659cb437a689019be20e4 Jul 28 17:09:22.556105 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:22.556034 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n776k" event={"ID":"39aa1164-b895-45e9-b64a-5220376fdd95","Type":"ContainerStarted","Data":"f7a22313a50a709ced0777e3086cc05a4fb7bc10554659cb437a689019be20e4"} Jul 28 17:09:27.587608 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:27.587569 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93b6b3b5-e860-4fee-9800-0f4534b72e67-metrics-certs\") pod \"network-metrics-daemon-5tdlw\" (UID: \"93b6b3b5-e860-4fee-9800-0f4534b72e67\") " pod="openshift-multus/network-metrics-daemon-5tdlw" Jul 28 17:09:27.591096 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:27.591070 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 28 17:09:27.599995 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:27.599974 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93b6b3b5-e860-4fee-9800-0f4534b72e67-metrics-certs\") pod \"network-metrics-daemon-5tdlw\" (UID: \"93b6b3b5-e860-4fee-9800-0f4534b72e67\") " pod="openshift-multus/network-metrics-daemon-5tdlw" Jul 28 17:09:27.689031 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:27.689006 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lzzz\" (UniqueName: \"kubernetes.io/projected/980a4184-ed46-42a1-99f3-86c49cfba43a-kube-api-access-7lzzz\") pod \"network-check-target-6dbsb\" (UID: \"980a4184-ed46-42a1-99f3-86c49cfba43a\") " pod="openshift-network-diagnostics/network-check-target-6dbsb" Jul 28 17:09:27.689133 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:27.689069 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9baa058e-a765-4c65-b5a6-ce0a7c022d97-original-pull-secret\") pod \"global-pull-secret-syncer-fbzsz\" (UID: \"9baa058e-a765-4c65-b5a6-ce0a7c022d97\") " pod="kube-system/global-pull-secret-syncer-fbzsz" Jul 28 17:09:27.691546 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:27.691521 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lzzz\" (UniqueName: \"kubernetes.io/projected/980a4184-ed46-42a1-99f3-86c49cfba43a-kube-api-access-7lzzz\") pod \"network-check-target-6dbsb\" (UID: \"980a4184-ed46-42a1-99f3-86c49cfba43a\") " pod="openshift-network-diagnostics/network-check-target-6dbsb" Jul 28 17:09:27.692294 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:27.692280 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 28 17:09:27.701916 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:27.701893 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9baa058e-a765-4c65-b5a6-ce0a7c022d97-original-pull-secret\") pod \"global-pull-secret-syncer-fbzsz\" (UID: \"9baa058e-a765-4c65-b5a6-ce0a7c022d97\") " pod="kube-system/global-pull-secret-syncer-fbzsz" Jul 28 17:09:27.727214 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:27.727189 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbzsz" Jul 28 17:09:27.735929 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:27.735900 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hr5cl\"" Jul 28 17:09:27.742812 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:27.742786 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8hnfw\"" Jul 28 17:09:27.742977 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:27.742935 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6dbsb" Jul 28 17:09:27.750158 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:27.750081 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tdlw" Jul 28 17:09:27.877615 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:27.877470 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fbzsz"] Jul 28 17:09:27.880244 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:09:27.880208 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9baa058e_a765_4c65_b5a6_ce0a7c022d97.slice/crio-0b2207c844ef00b7672397985fdc440809370b31cdc9622bb61e30acbbacadfa WatchSource:0}: Error finding container 0b2207c844ef00b7672397985fdc440809370b31cdc9622bb61e30acbbacadfa: Status 404 returned error can't find the container with id 0b2207c844ef00b7672397985fdc440809370b31cdc9622bb61e30acbbacadfa Jul 28 17:09:27.891615 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:27.891596 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5tdlw"] Jul 28 17:09:27.893640 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:09:27.893615 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93b6b3b5_e860_4fee_9800_0f4534b72e67.slice/crio-c3ece961265dc3380c49ded4ce1f4eaca810f1b9c5fb0a1ec353b74a79523bb1 WatchSource:0}: Error finding container c3ece961265dc3380c49ded4ce1f4eaca810f1b9c5fb0a1ec353b74a79523bb1: Status 404 returned error can't find the container with id c3ece961265dc3380c49ded4ce1f4eaca810f1b9c5fb0a1ec353b74a79523bb1 Jul 28 17:09:27.907541 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:27.907527 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6dbsb"] Jul 28 17:09:27.909457 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:09:27.909438 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod980a4184_ed46_42a1_99f3_86c49cfba43a.slice/crio-25885d0c4abe8f23e0cd93f9a4f3230bd3c7fd59cbc8a8ecb213ec7b87e261c0 WatchSource:0}: Error finding container 25885d0c4abe8f23e0cd93f9a4f3230bd3c7fd59cbc8a8ecb213ec7b87e261c0: Status 404 returned error can't find the container with id 25885d0c4abe8f23e0cd93f9a4f3230bd3c7fd59cbc8a8ecb213ec7b87e261c0 Jul 28 17:09:28.581488 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:28.581442 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fbzsz" event={"ID":"9baa058e-a765-4c65-b5a6-ce0a7c022d97","Type":"ContainerStarted","Data":"0b2207c844ef00b7672397985fdc440809370b31cdc9622bb61e30acbbacadfa"} Jul 28 17:09:28.583437 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:28.583372 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5tdlw" event={"ID":"93b6b3b5-e860-4fee-9800-0f4534b72e67","Type":"ContainerStarted","Data":"c3ece961265dc3380c49ded4ce1f4eaca810f1b9c5fb0a1ec353b74a79523bb1"} Jul 28 17:09:28.585646 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:28.585583 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n776k" event={"ID":"39aa1164-b895-45e9-b64a-5220376fdd95","Type":"ContainerStarted","Data":"6fa3d2225a54692edb5c5dbab1bbc4a3795d0fd5c92218c3c74e2a2ad8306617"} Jul 28 17:09:28.588672 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:28.588645 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6dbsb" event={"ID":"980a4184-ed46-42a1-99f3-86c49cfba43a","Type":"ContainerStarted","Data":"e7bb4d6bd49115c8ed1ab1e47d8ff82b96b90ff8278a9a10c4f3560cc7255a83"} Jul 28 17:09:28.589058 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:28.588694 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6dbsb" event={"ID":"980a4184-ed46-42a1-99f3-86c49cfba43a","Type":"ContainerStarted","Data":"25885d0c4abe8f23e0cd93f9a4f3230bd3c7fd59cbc8a8ecb213ec7b87e261c0"} Jul 28 17:09:28.589198 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:28.589126 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-6dbsb" Jul 28 17:09:28.613694 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:28.613618 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-6dbsb" podStartSLOduration=66.613597576 podStartE2EDuration="1m6.613597576s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:08:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:28.611643423 +0000 UTC m=+67.220574793" watchObservedRunningTime="2026-07-28 17:09:28.613597576 +0000 UTC m=+67.222528948" Jul 28 17:09:28.699776 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:28.699733 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fd91b2e7-ff77-474f-a69b-47a84fe174a3-metrics-tls\") pod \"dns-default-49xwx\" (UID: \"fd91b2e7-ff77-474f-a69b-47a84fe174a3\") " pod="openshift-dns/dns-default-49xwx" Jul 28 17:09:28.699933 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:28.699831 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-registry-tls\") pod \"image-registry-6b5f57f94b-vgwb7\" (UID: \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\") " pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" Jul 28 17:09:28.699933 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:28.699869 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af095ddd-e1cd-4148-9796-d60be4fef0e2-metrics-certs\") pod \"router-default-64c477cb5c-9stkr\" (UID: \"af095ddd-e1cd-4148-9796-d60be4fef0e2\") " pod="openshift-ingress/router-default-64c477cb5c-9stkr" Jul 28 17:09:28.700634 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:28.700055 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af095ddd-e1cd-4148-9796-d60be4fef0e2-service-ca-bundle\") pod \"router-default-64c477cb5c-9stkr\" (UID: \"af095ddd-e1cd-4148-9796-d60be4fef0e2\") " pod="openshift-ingress/router-default-64c477cb5c-9stkr" Jul 28 17:09:28.700634 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:28.700150 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb5065a0-5a41-40db-a798-5ccd320bdefe-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-sbz9k\" (UID: \"fb5065a0-5a41-40db-a798-5ccd320bdefe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-sbz9k" Jul 28 17:09:28.700634 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:28.700205 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c6ad5592-e8fc-41e7-8fcf-caa57c3eae35-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-m4bxm\" (UID: \"c6ad5592-e8fc-41e7-8fcf-caa57c3eae35\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-m4bxm" Jul 28 17:09:28.700634 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:28.700244 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/52730019-2aad-4136-91b5-1b82eb52e0d1-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-ngj99\" (UID: \"52730019-2aad-4136-91b5-1b82eb52e0d1\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-ngj99" Jul 28 17:09:28.700918 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:28.700881 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af095ddd-e1cd-4148-9796-d60be4fef0e2-service-ca-bundle\") pod \"router-default-64c477cb5c-9stkr\" (UID: \"af095ddd-e1cd-4148-9796-d60be4fef0e2\") " pod="openshift-ingress/router-default-64c477cb5c-9stkr" Jul 28 17:09:28.702916 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:28.702873 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fd91b2e7-ff77-474f-a69b-47a84fe174a3-metrics-tls\") pod \"dns-default-49xwx\" (UID: \"fd91b2e7-ff77-474f-a69b-47a84fe174a3\") " pod="openshift-dns/dns-default-49xwx" Jul 28 17:09:28.703491 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:28.703452 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af095ddd-e1cd-4148-9796-d60be4fef0e2-metrics-certs\") pod \"router-default-64c477cb5c-9stkr\" (UID: \"af095ddd-e1cd-4148-9796-d60be4fef0e2\") " pod="openshift-ingress/router-default-64c477cb5c-9stkr" Jul 28 17:09:28.704777 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:28.704725 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c6ad5592-e8fc-41e7-8fcf-caa57c3eae35-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-m4bxm\" (UID: \"c6ad5592-e8fc-41e7-8fcf-caa57c3eae35\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-m4bxm" Jul 28 17:09:28.705436 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:28.705386 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-registry-tls\") pod \"image-registry-6b5f57f94b-vgwb7\" (UID: \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\") " pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" Jul 28 17:09:28.705977 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:28.705916 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb5065a0-5a41-40db-a798-5ccd320bdefe-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-sbz9k\" (UID: \"fb5065a0-5a41-40db-a798-5ccd320bdefe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-sbz9k" Jul 28 17:09:28.706535 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:28.706515 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/52730019-2aad-4136-91b5-1b82eb52e0d1-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-ngj99\" (UID: \"52730019-2aad-4136-91b5-1b82eb52e0d1\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-ngj99" Jul 28 17:09:28.841585 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:28.841507 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-d5n6c\"" Jul 28 17:09:28.849606 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:28.849365 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-ngj99" Jul 28 17:09:28.880326 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:28.880128 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6dhdb\"" Jul 28 17:09:28.888889 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:28.888737 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" Jul 28 17:09:28.889007 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:28.888980 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-hdkhn\"" Jul 28 17:09:28.895862 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:28.895830 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-sbz9k" Jul 28 17:09:28.900344 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:28.900313 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-5c7bt\"" Jul 28 17:09:28.907321 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:28.907300 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-c6hl9\"" Jul 28 17:09:28.908658 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:28.908608 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-m4bxm" Jul 28 17:09:28.915091 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:28.914993 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-64c477cb5c-9stkr" Jul 28 17:09:28.928206 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:28.928008 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pz4fk\"" Jul 28 17:09:28.936206 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:28.936179 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-49xwx" Jul 28 17:09:29.041687 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:29.041652 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-ngj99"] Jul 28 17:09:29.152286 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:29.152202 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b5f57f94b-vgwb7"] Jul 28 17:09:29.160556 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:09:29.160474 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d8be12f_23ea_4ddf_bc6d_9bb0cf9da50c.slice/crio-76628011d9cee26fb33c5a8f14a2234906931680ca7fff0be9e593e9d8b9b649 WatchSource:0}: Error finding container 76628011d9cee26fb33c5a8f14a2234906931680ca7fff0be9e593e9d8b9b649: Status 404 returned error can't find the container with id 76628011d9cee26fb33c5a8f14a2234906931680ca7fff0be9e593e9d8b9b649 Jul 28 17:09:29.168263 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:29.167959 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-sbz9k"] Jul 28 17:09:29.218893 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:29.218812 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-49xwx"] Jul 28 17:09:29.234018 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:09:29.233987 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd91b2e7_ff77_474f_a69b_47a84fe174a3.slice/crio-9da0424728926b319b666eaebe0a6b177de74f619c2c2b979963b178e78f1011 WatchSource:0}: Error finding container 9da0424728926b319b666eaebe0a6b177de74f619c2c2b979963b178e78f1011: Status 404 returned error can't find the container with id 9da0424728926b319b666eaebe0a6b177de74f619c2c2b979963b178e78f1011 Jul 28 17:09:29.245063 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:29.244997 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-m4bxm"] Jul 28 17:09:29.255545 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:09:29.255513 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6ad5592_e8fc_41e7_8fcf_caa57c3eae35.slice/crio-6dd5f25489ef9f53dc90ce7a8f6ff944157dd41b628ba8476fc190495cedafda WatchSource:0}: Error finding container 6dd5f25489ef9f53dc90ce7a8f6ff944157dd41b628ba8476fc190495cedafda: Status 404 returned error can't find the container with id 6dd5f25489ef9f53dc90ce7a8f6ff944157dd41b628ba8476fc190495cedafda Jul 28 17:09:29.277688 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:29.277472 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-64c477cb5c-9stkr"] Jul 28 17:09:29.282392 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:09:29.282365 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf095ddd_e1cd_4148_9796_d60be4fef0e2.slice/crio-dbb4f146180c98c1d0c3266a502bc5ee966671ec7e2d1f0fb7ee8548effc013a WatchSource:0}: Error finding container dbb4f146180c98c1d0c3266a502bc5ee966671ec7e2d1f0fb7ee8548effc013a: Status 404 returned error can't find the container with id dbb4f146180c98c1d0c3266a502bc5ee966671ec7e2d1f0fb7ee8548effc013a Jul 28 17:09:29.598715 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:29.598577 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-ngj99" event={"ID":"52730019-2aad-4136-91b5-1b82eb52e0d1","Type":"ContainerStarted","Data":"d8fac92f92b3a507333b31f29265e346aeb082384b518a6d68cc79934f014722"} Jul 28 17:09:29.601250 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:29.601172 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-49xwx" event={"ID":"fd91b2e7-ff77-474f-a69b-47a84fe174a3","Type":"ContainerStarted","Data":"9da0424728926b319b666eaebe0a6b177de74f619c2c2b979963b178e78f1011"} Jul 28 17:09:29.603757 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:29.603706 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-m4bxm" event={"ID":"c6ad5592-e8fc-41e7-8fcf-caa57c3eae35","Type":"ContainerStarted","Data":"6dd5f25489ef9f53dc90ce7a8f6ff944157dd41b628ba8476fc190495cedafda"} Jul 28 17:09:29.607174 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:29.606389 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-64c477cb5c-9stkr" event={"ID":"af095ddd-e1cd-4148-9796-d60be4fef0e2","Type":"ContainerStarted","Data":"4cb9d28d9f815619eb291028ee2cd62cffc45f135c0fdcf5b7105d87248aec3c"} Jul 28 17:09:29.607174 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:29.606431 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-64c477cb5c-9stkr" event={"ID":"af095ddd-e1cd-4148-9796-d60be4fef0e2","Type":"ContainerStarted","Data":"dbb4f146180c98c1d0c3266a502bc5ee966671ec7e2d1f0fb7ee8548effc013a"} Jul 28 17:09:29.610493 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:29.610375 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-sbz9k" event={"ID":"fb5065a0-5a41-40db-a798-5ccd320bdefe","Type":"ContainerStarted","Data":"8efbec77004939054bd73224b71351ee4ae1f3bc34a9b2dc570cba3f39ba7c38"} Jul 28 17:09:29.613916 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:29.613843 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" event={"ID":"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c","Type":"ContainerStarted","Data":"6355a36781abcf2ed3e6e6b468b9cd89daaa0b4cdb5b59e06028fe0b3ee78fcb"} Jul 28 17:09:29.613916 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:29.613888 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" event={"ID":"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c","Type":"ContainerStarted","Data":"76628011d9cee26fb33c5a8f14a2234906931680ca7fff0be9e593e9d8b9b649"} Jul 28 17:09:29.614489 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:29.614438 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" Jul 28 17:09:29.649772 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:29.649718 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-64c477cb5c-9stkr" podStartSLOduration=57.64969743 podStartE2EDuration="57.64969743s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:08:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:29.639392897 +0000 UTC m=+68.248324292" watchObservedRunningTime="2026-07-28 17:09:29.64969743 +0000 UTC m=+68.258628801" Jul 28 17:09:29.915820 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:29.915672 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-64c477cb5c-9stkr" Jul 28 17:09:29.918961 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:29.918928 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-64c477cb5c-9stkr" Jul 28 17:09:29.945073 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:29.944041 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" podStartSLOduration=57.944018327 podStartE2EDuration="57.944018327s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:08:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:29.695546769 +0000 UTC m=+68.304478140" watchObservedRunningTime="2026-07-28 17:09:29.944018327 +0000 UTC m=+68.552949700" Jul 28 17:09:30.622374 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:30.619950 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-64c477cb5c-9stkr" Jul 28 17:09:30.622374 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:30.621223 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-64c477cb5c-9stkr" Jul 28 17:09:30.734383 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:30.734345 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-859f88d764-48mxj"] Jul 28 17:09:30.755543 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:30.754900 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b6bb8dccd-pcjfq"] Jul 28 17:09:30.756191 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:30.755803 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-859f88d764-48mxj" Jul 28 17:09:30.763188 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:30.761278 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 28 17:09:30.763188 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:30.761602 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 28 17:09:30.763188 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:30.761803 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 28 17:09:30.763188 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:30.761987 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 28 17:09:30.778760 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:30.777251 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6b5f57f94b-vgwb7"] Jul 28 17:09:30.778760 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:30.777310 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b6bb8dccd-pcjfq"] Jul 28 17:09:30.778760 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:30.777328 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-859f88d764-48mxj"] Jul 28 17:09:30.778760 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:30.777433 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b6bb8dccd-pcjfq" Jul 28 17:09:30.785715 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:30.785264 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-ctr7j\"" Jul 28 17:09:30.785715 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:30.785657 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 28 17:09:30.835044 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:30.833827 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7b5446bd95-ngjw4"] Jul 28 17:09:30.860724 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:30.860528 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b5446bd95-ngjw4" Jul 28 17:09:30.867999 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:30.867968 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b5446bd95-ngjw4"] Jul 28 17:09:30.923874 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:30.923788 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbhmk\" (UniqueName: \"kubernetes.io/projected/2b8a1db7-5a30-4ca1-9dcb-4b1c3b00fca0-kube-api-access-lbhmk\") pod \"klusterlet-addon-workmgr-859f88d764-48mxj\" (UID: \"2b8a1db7-5a30-4ca1-9dcb-4b1c3b00fca0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-859f88d764-48mxj" Jul 28 17:09:30.924041 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:30.923885 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5q4fz\" (UniqueName: \"kubernetes.io/projected/4f352e98-8159-4205-9883-c4f945fe9e94-kube-api-access-5q4fz\") pod \"managed-serviceaccount-addon-agent-5b6bb8dccd-pcjfq\" (UID: \"4f352e98-8159-4205-9883-c4f945fe9e94\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b6bb8dccd-pcjfq" Jul 28 17:09:30.924041 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:30.923937 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4f352e98-8159-4205-9883-c4f945fe9e94-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5b6bb8dccd-pcjfq\" (UID: \"4f352e98-8159-4205-9883-c4f945fe9e94\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b6bb8dccd-pcjfq" Jul 28 17:09:30.924041 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:30.923992 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b8a1db7-5a30-4ca1-9dcb-4b1c3b00fca0-tmp\") pod \"klusterlet-addon-workmgr-859f88d764-48mxj\" (UID: \"2b8a1db7-5a30-4ca1-9dcb-4b1c3b00fca0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-859f88d764-48mxj" Jul 28 17:09:30.924462 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:30.924049 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/2b8a1db7-5a30-4ca1-9dcb-4b1c3b00fca0-klusterlet-config\") pod \"klusterlet-addon-workmgr-859f88d764-48mxj\" (UID: \"2b8a1db7-5a30-4ca1-9dcb-4b1c3b00fca0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-859f88d764-48mxj" Jul 28 17:09:31.024833 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:31.024794 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4f352e98-8159-4205-9883-c4f945fe9e94-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5b6bb8dccd-pcjfq\" (UID: \"4f352e98-8159-4205-9883-c4f945fe9e94\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b6bb8dccd-pcjfq" Jul 28 17:09:31.025067 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:31.024854 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5qs7\" (UniqueName: \"kubernetes.io/projected/0aa92363-6609-459f-9ce0-bbc1700a8bd8-kube-api-access-s5qs7\") pod \"image-registry-7b5446bd95-ngjw4\" (UID: \"0aa92363-6609-459f-9ce0-bbc1700a8bd8\") " pod="openshift-image-registry/image-registry-7b5446bd95-ngjw4" Jul 28 17:09:31.025067 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:31.024889 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0aa92363-6609-459f-9ce0-bbc1700a8bd8-registry-tls\") pod \"image-registry-7b5446bd95-ngjw4\" (UID: \"0aa92363-6609-459f-9ce0-bbc1700a8bd8\") " pod="openshift-image-registry/image-registry-7b5446bd95-ngjw4" Jul 28 17:09:31.025067 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:31.024923 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0aa92363-6609-459f-9ce0-bbc1700a8bd8-ca-trust-extracted\") pod \"image-registry-7b5446bd95-ngjw4\" (UID: \"0aa92363-6609-459f-9ce0-bbc1700a8bd8\") " pod="openshift-image-registry/image-registry-7b5446bd95-ngjw4" Jul 28 17:09:31.025067 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:31.024956 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0aa92363-6609-459f-9ce0-bbc1700a8bd8-registry-certificates\") pod \"image-registry-7b5446bd95-ngjw4\" (UID: \"0aa92363-6609-459f-9ce0-bbc1700a8bd8\") " pod="openshift-image-registry/image-registry-7b5446bd95-ngjw4" Jul 28 17:09:31.025067 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:31.024985 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b8a1db7-5a30-4ca1-9dcb-4b1c3b00fca0-tmp\") pod \"klusterlet-addon-workmgr-859f88d764-48mxj\" (UID: \"2b8a1db7-5a30-4ca1-9dcb-4b1c3b00fca0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-859f88d764-48mxj" Jul 28 17:09:31.025067 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:31.025009 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0aa92363-6609-459f-9ce0-bbc1700a8bd8-trusted-ca\") pod \"image-registry-7b5446bd95-ngjw4\" (UID: \"0aa92363-6609-459f-9ce0-bbc1700a8bd8\") " pod="openshift-image-registry/image-registry-7b5446bd95-ngjw4" Jul 28 17:09:31.025067 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:31.025036 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/2b8a1db7-5a30-4ca1-9dcb-4b1c3b00fca0-klusterlet-config\") pod \"klusterlet-addon-workmgr-859f88d764-48mxj\" (UID: \"2b8a1db7-5a30-4ca1-9dcb-4b1c3b00fca0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-859f88d764-48mxj" Jul 28 17:09:31.025067 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:31.025066 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbhmk\" (UniqueName: \"kubernetes.io/projected/2b8a1db7-5a30-4ca1-9dcb-4b1c3b00fca0-kube-api-access-lbhmk\") pod \"klusterlet-addon-workmgr-859f88d764-48mxj\" (UID: \"2b8a1db7-5a30-4ca1-9dcb-4b1c3b00fca0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-859f88d764-48mxj" Jul 28 17:09:31.025512 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:31.025093 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0aa92363-6609-459f-9ce0-bbc1700a8bd8-installation-pull-secrets\") pod \"image-registry-7b5446bd95-ngjw4\" (UID: \"0aa92363-6609-459f-9ce0-bbc1700a8bd8\") " pod="openshift-image-registry/image-registry-7b5446bd95-ngjw4" Jul 28 17:09:31.026347 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:31.025780 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0aa92363-6609-459f-9ce0-bbc1700a8bd8-bound-sa-token\") pod \"image-registry-7b5446bd95-ngjw4\" (UID: \"0aa92363-6609-459f-9ce0-bbc1700a8bd8\") " pod="openshift-image-registry/image-registry-7b5446bd95-ngjw4" Jul 28 17:09:31.026347 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:31.025842 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q4fz\" (UniqueName: \"kubernetes.io/projected/4f352e98-8159-4205-9883-c4f945fe9e94-kube-api-access-5q4fz\") pod \"managed-serviceaccount-addon-agent-5b6bb8dccd-pcjfq\" (UID: \"4f352e98-8159-4205-9883-c4f945fe9e94\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b6bb8dccd-pcjfq" Jul 28 17:09:31.026347 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:31.025939 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b8a1db7-5a30-4ca1-9dcb-4b1c3b00fca0-tmp\") pod \"klusterlet-addon-workmgr-859f88d764-48mxj\" (UID: \"2b8a1db7-5a30-4ca1-9dcb-4b1c3b00fca0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-859f88d764-48mxj" Jul 28 17:09:31.026347 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:31.026178 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0aa92363-6609-459f-9ce0-bbc1700a8bd8-image-registry-private-configuration\") pod \"image-registry-7b5446bd95-ngjw4\" (UID: \"0aa92363-6609-459f-9ce0-bbc1700a8bd8\") " pod="openshift-image-registry/image-registry-7b5446bd95-ngjw4" Jul 28 17:09:31.031326 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:31.031268 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/2b8a1db7-5a30-4ca1-9dcb-4b1c3b00fca0-klusterlet-config\") pod \"klusterlet-addon-workmgr-859f88d764-48mxj\" (UID: \"2b8a1db7-5a30-4ca1-9dcb-4b1c3b00fca0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-859f88d764-48mxj" Jul 28 17:09:31.036167 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:31.036085 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4f352e98-8159-4205-9883-c4f945fe9e94-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5b6bb8dccd-pcjfq\" (UID: \"4f352e98-8159-4205-9883-c4f945fe9e94\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b6bb8dccd-pcjfq" Jul 28 17:09:31.049935 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:31.049887 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5q4fz\" (UniqueName: \"kubernetes.io/projected/4f352e98-8159-4205-9883-c4f945fe9e94-kube-api-access-5q4fz\") pod \"managed-serviceaccount-addon-agent-5b6bb8dccd-pcjfq\" (UID: \"4f352e98-8159-4205-9883-c4f945fe9e94\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b6bb8dccd-pcjfq" Jul 28 17:09:31.057988 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:31.057917 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbhmk\" (UniqueName: \"kubernetes.io/projected/2b8a1db7-5a30-4ca1-9dcb-4b1c3b00fca0-kube-api-access-lbhmk\") pod \"klusterlet-addon-workmgr-859f88d764-48mxj\" (UID: \"2b8a1db7-5a30-4ca1-9dcb-4b1c3b00fca0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-859f88d764-48mxj" Jul 28 17:09:31.073083 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:31.073056 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-859f88d764-48mxj" Jul 28 17:09:31.104802 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:31.104778 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b6bb8dccd-pcjfq" Jul 28 17:09:31.126677 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:31.126647 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0aa92363-6609-459f-9ce0-bbc1700a8bd8-installation-pull-secrets\") pod \"image-registry-7b5446bd95-ngjw4\" (UID: \"0aa92363-6609-459f-9ce0-bbc1700a8bd8\") " pod="openshift-image-registry/image-registry-7b5446bd95-ngjw4" Jul 28 17:09:31.126791 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:31.126722 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0aa92363-6609-459f-9ce0-bbc1700a8bd8-bound-sa-token\") pod \"image-registry-7b5446bd95-ngjw4\" (UID: \"0aa92363-6609-459f-9ce0-bbc1700a8bd8\") " pod="openshift-image-registry/image-registry-7b5446bd95-ngjw4" Jul 28 17:09:31.126791 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:31.126769 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0aa92363-6609-459f-9ce0-bbc1700a8bd8-image-registry-private-configuration\") pod \"image-registry-7b5446bd95-ngjw4\" (UID: \"0aa92363-6609-459f-9ce0-bbc1700a8bd8\") " pod="openshift-image-registry/image-registry-7b5446bd95-ngjw4" Jul 28 17:09:31.126905 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:31.126806 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5qs7\" (UniqueName: \"kubernetes.io/projected/0aa92363-6609-459f-9ce0-bbc1700a8bd8-kube-api-access-s5qs7\") pod \"image-registry-7b5446bd95-ngjw4\" (UID: \"0aa92363-6609-459f-9ce0-bbc1700a8bd8\") " pod="openshift-image-registry/image-registry-7b5446bd95-ngjw4" Jul 28 17:09:31.126905 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:31.126834 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0aa92363-6609-459f-9ce0-bbc1700a8bd8-registry-tls\") pod \"image-registry-7b5446bd95-ngjw4\" (UID: \"0aa92363-6609-459f-9ce0-bbc1700a8bd8\") " pod="openshift-image-registry/image-registry-7b5446bd95-ngjw4" Jul 28 17:09:31.126905 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:31.126871 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0aa92363-6609-459f-9ce0-bbc1700a8bd8-ca-trust-extracted\") pod \"image-registry-7b5446bd95-ngjw4\" (UID: \"0aa92363-6609-459f-9ce0-bbc1700a8bd8\") " pod="openshift-image-registry/image-registry-7b5446bd95-ngjw4" Jul 28 17:09:31.127054 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:31.126903 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0aa92363-6609-459f-9ce0-bbc1700a8bd8-registry-certificates\") pod \"image-registry-7b5446bd95-ngjw4\" (UID: \"0aa92363-6609-459f-9ce0-bbc1700a8bd8\") " pod="openshift-image-registry/image-registry-7b5446bd95-ngjw4" Jul 28 17:09:31.127054 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:31.126930 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0aa92363-6609-459f-9ce0-bbc1700a8bd8-trusted-ca\") pod \"image-registry-7b5446bd95-ngjw4\" (UID: \"0aa92363-6609-459f-9ce0-bbc1700a8bd8\") " pod="openshift-image-registry/image-registry-7b5446bd95-ngjw4" Jul 28 17:09:31.128918 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:31.127793 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0aa92363-6609-459f-9ce0-bbc1700a8bd8-ca-trust-extracted\") pod \"image-registry-7b5446bd95-ngjw4\" (UID: \"0aa92363-6609-459f-9ce0-bbc1700a8bd8\") " pod="openshift-image-registry/image-registry-7b5446bd95-ngjw4" Jul 28 17:09:31.128918 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:31.128188 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0aa92363-6609-459f-9ce0-bbc1700a8bd8-trusted-ca\") pod \"image-registry-7b5446bd95-ngjw4\" (UID: \"0aa92363-6609-459f-9ce0-bbc1700a8bd8\") " pod="openshift-image-registry/image-registry-7b5446bd95-ngjw4" Jul 28 17:09:31.128918 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:31.128572 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0aa92363-6609-459f-9ce0-bbc1700a8bd8-registry-certificates\") pod \"image-registry-7b5446bd95-ngjw4\" (UID: \"0aa92363-6609-459f-9ce0-bbc1700a8bd8\") " pod="openshift-image-registry/image-registry-7b5446bd95-ngjw4" Jul 28 17:09:31.132023 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:31.131901 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0aa92363-6609-459f-9ce0-bbc1700a8bd8-registry-tls\") pod \"image-registry-7b5446bd95-ngjw4\" (UID: \"0aa92363-6609-459f-9ce0-bbc1700a8bd8\") " pod="openshift-image-registry/image-registry-7b5446bd95-ngjw4" Jul 28 17:09:31.132311 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:31.132287 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0aa92363-6609-459f-9ce0-bbc1700a8bd8-installation-pull-secrets\") pod \"image-registry-7b5446bd95-ngjw4\" (UID: \"0aa92363-6609-459f-9ce0-bbc1700a8bd8\") " pod="openshift-image-registry/image-registry-7b5446bd95-ngjw4" Jul 28 17:09:31.134021 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:31.133994 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0aa92363-6609-459f-9ce0-bbc1700a8bd8-image-registry-private-configuration\") pod \"image-registry-7b5446bd95-ngjw4\" (UID: \"0aa92363-6609-459f-9ce0-bbc1700a8bd8\") " pod="openshift-image-registry/image-registry-7b5446bd95-ngjw4" Jul 28 17:09:31.139626 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:31.139603 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5qs7\" (UniqueName: \"kubernetes.io/projected/0aa92363-6609-459f-9ce0-bbc1700a8bd8-kube-api-access-s5qs7\") pod \"image-registry-7b5446bd95-ngjw4\" (UID: \"0aa92363-6609-459f-9ce0-bbc1700a8bd8\") " pod="openshift-image-registry/image-registry-7b5446bd95-ngjw4" Jul 28 17:09:31.140543 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:31.140502 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0aa92363-6609-459f-9ce0-bbc1700a8bd8-bound-sa-token\") pod \"image-registry-7b5446bd95-ngjw4\" (UID: \"0aa92363-6609-459f-9ce0-bbc1700a8bd8\") " pod="openshift-image-registry/image-registry-7b5446bd95-ngjw4" Jul 28 17:09:31.174981 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:31.174891 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b5446bd95-ngjw4" Jul 28 17:09:37.412966 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:37.412893 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-859f88d764-48mxj"] Jul 28 17:09:37.549184 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:37.549088 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b5446bd95-ngjw4"] Jul 28 17:09:37.555384 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:09:37.555355 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0aa92363_6609_459f_9ce0_bbc1700a8bd8.slice/crio-9d26d50e6313855a01ffcf137ef3a9f5a90c260bf847ec68561320e0ba51a4df WatchSource:0}: Error finding container 9d26d50e6313855a01ffcf137ef3a9f5a90c260bf847ec68561320e0ba51a4df: Status 404 returned error can't find the container with id 9d26d50e6313855a01ffcf137ef3a9f5a90c260bf847ec68561320e0ba51a4df Jul 28 17:09:37.573105 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:37.571106 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b6bb8dccd-pcjfq"] Jul 28 17:09:37.593252 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:09:37.591019 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4f352e98_8159_4205_9883_c4f945fe9e94.slice/crio-721b87e305d89c56217ccb22456b362986a81db6c48032526b8b0e01380859b5 WatchSource:0}: Error finding container 721b87e305d89c56217ccb22456b362986a81db6c48032526b8b0e01380859b5: Status 404 returned error can't find the container with id 721b87e305d89c56217ccb22456b362986a81db6c48032526b8b0e01380859b5 Jul 28 17:09:37.652559 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:37.652517 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-sbz9k" event={"ID":"fb5065a0-5a41-40db-a798-5ccd320bdefe","Type":"ContainerStarted","Data":"9078ea77bb999ab21d232eaacecba80161f2c0f44f20bdc8d3139283a8df211f"} Jul 28 17:09:37.653979 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:37.653875 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b5446bd95-ngjw4" event={"ID":"0aa92363-6609-459f-9ce0-bbc1700a8bd8","Type":"ContainerStarted","Data":"9d26d50e6313855a01ffcf137ef3a9f5a90c260bf847ec68561320e0ba51a4df"} Jul 28 17:09:37.656609 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:37.656520 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-49xwx" event={"ID":"fd91b2e7-ff77-474f-a69b-47a84fe174a3","Type":"ContainerStarted","Data":"f28062be04f336abce6516ae0ee2b7e46cf6c2b77484d293e2fbd91fae796bfe"} Jul 28 17:09:37.667161 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:37.666644 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fbzsz" event={"ID":"9baa058e-a765-4c65-b5a6-ce0a7c022d97","Type":"ContainerStarted","Data":"82399860052b53d85785451973b53e1d4e2b9c6adb4c26713eecdde67448519b"} Jul 28 17:09:37.670153 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:37.669461 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-m4bxm" event={"ID":"c6ad5592-e8fc-41e7-8fcf-caa57c3eae35","Type":"ContainerStarted","Data":"d65c8cb50d581428f52f191a286d8b473715f5625d1ec993dcfb98ce289b452b"} Jul 28 17:09:37.672987 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:37.671930 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-859f88d764-48mxj" event={"ID":"2b8a1db7-5a30-4ca1-9dcb-4b1c3b00fca0","Type":"ContainerStarted","Data":"158a27cc8c5d2b4df6b18852b54aaf12f6aa8fd9bc96d0cb97fd403288587e05"} Jul 28 17:09:37.680356 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:37.679996 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-ngj99" event={"ID":"52730019-2aad-4136-91b5-1b82eb52e0d1","Type":"ContainerStarted","Data":"f17d0110d9ba211efa35e7bd433d8b90ae64512b404f4b97fcd2235833744df8"} Jul 28 17:09:37.700156 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:37.696632 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b6bb8dccd-pcjfq" event={"ID":"4f352e98-8159-4205-9883-c4f945fe9e94","Type":"ContainerStarted","Data":"721b87e305d89c56217ccb22456b362986a81db6c48032526b8b0e01380859b5"} Jul 28 17:09:37.704148 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:37.703449 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5tdlw" event={"ID":"93b6b3b5-e860-4fee-9800-0f4534b72e67","Type":"ContainerStarted","Data":"cd1b57409cea3cb79f5c6c467f2557011e94c9fb58ff5233f741133001343752"} Jul 28 17:09:37.718078 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:37.717988 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n776k" event={"ID":"39aa1164-b895-45e9-b64a-5220376fdd95","Type":"ContainerStarted","Data":"6063f95ffc48358357d66c171cdca5e840b68a5790ab3cd31da8f73b7ae1f676"} Jul 28 17:09:37.722932 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:37.722696 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-fbzsz" podStartSLOduration=65.251868018 podStartE2EDuration="1m14.722677553s" totalImagesPullingTime="9.470809535s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:08:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:37.69172665 +0000 UTC m=+76.300658021" watchObservedRunningTime="2026-07-28 17:09:37.722677553 +0000 UTC m=+76.331608948" Jul 28 17:09:37.755969 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:37.755418 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-ngj99" podStartSLOduration=59.161672891 podStartE2EDuration="1m5.755396517s" totalImagesPullingTime="6.593723626s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:08:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:37.729375427 +0000 UTC m=+76.338306798" watchObservedRunningTime="2026-07-28 17:09:37.755396517 +0000 UTC m=+76.364327888" Jul 28 17:09:37.781151 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:37.781042 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-5tdlw" podStartSLOduration=68.82475008 podStartE2EDuration="1m15.781021192s" totalImagesPullingTime="6.956271112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:08:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:37.779276771 +0000 UTC m=+76.388208141" watchObservedRunningTime="2026-07-28 17:09:37.781021192 +0000 UTC m=+76.389952564" Jul 28 17:09:37.781416 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:37.781384 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-m4bxm" podStartSLOduration=57.595840486 podStartE2EDuration="1m5.781369402s" totalImagesPullingTime="8.185528916s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:08:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:37.755794812 +0000 UTC m=+76.364726183" watchObservedRunningTime="2026-07-28 17:09:37.781369402 +0000 UTC m=+76.390300772" Jul 28 17:09:38.731616 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:38.731537 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5tdlw" event={"ID":"93b6b3b5-e860-4fee-9800-0f4534b72e67","Type":"ContainerStarted","Data":"87f498934d8a2e610384ef688ed6bd96c8e65334fa4d179e0c4a2f7c3da43eae"} Jul 28 17:09:38.736649 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:38.736615 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-sbz9k" event={"ID":"fb5065a0-5a41-40db-a798-5ccd320bdefe","Type":"ContainerStarted","Data":"6a1aa8826d1d450ec6df943eb587949ffd419bb39b8594e00eafe56554110168"} Jul 28 17:09:38.742560 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:38.740954 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b5446bd95-ngjw4" event={"ID":"0aa92363-6609-459f-9ce0-bbc1700a8bd8","Type":"ContainerStarted","Data":"711f4b4996fd1c4658e9e1168c4809d5704682f6f42302911abdaeddd8c982b3"} Jul 28 17:09:38.742560 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:38.741512 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7b5446bd95-ngjw4" Jul 28 17:09:38.746908 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:38.746068 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-49xwx" event={"ID":"fd91b2e7-ff77-474f-a69b-47a84fe174a3","Type":"ContainerStarted","Data":"78505c9c11241b4cee3e27eb1c95b48a720b455f3c3c0eb8dcc6b8681c3ef959"} Jul 28 17:09:38.746908 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:38.746134 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-49xwx" Jul 28 17:09:38.761266 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:38.761217 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-sbz9k" podStartSLOduration=58.659335126 podStartE2EDuration="1m6.761199017s" totalImagesPullingTime="8.101863891s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:08:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:38.760002896 +0000 UTC m=+77.368934267" watchObservedRunningTime="2026-07-28 17:09:38.761199017 +0000 UTC m=+77.370130388" Jul 28 17:09:38.797155 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:38.795843 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7b5446bd95-ngjw4" podStartSLOduration=8.79582248 podStartE2EDuration="8.79582248s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:09:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:38.79366144 +0000 UTC m=+77.402592811" watchObservedRunningTime="2026-07-28 17:09:38.79582248 +0000 UTC m=+77.404753851" Jul 28 17:09:38.822712 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:38.822314 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-49xwx" podStartSLOduration=34.694658038 podStartE2EDuration="42.822292994s" totalImagesPullingTime="8.127634956s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:08:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:38.820615677 +0000 UTC m=+77.429547046" watchObservedRunningTime="2026-07-28 17:09:38.822292994 +0000 UTC m=+77.431224365" Jul 28 17:09:43.770239 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:43.770144 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-859f88d764-48mxj" event={"ID":"2b8a1db7-5a30-4ca1-9dcb-4b1c3b00fca0","Type":"ContainerStarted","Data":"e4cac2775f5e4df3acc6c549cd2c0b6a6a16c52a90a605215499444513e9d6e1"} Jul 28 17:09:43.770649 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:43.770302 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-859f88d764-48mxj" Jul 28 17:09:43.772002 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:43.771973 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b6bb8dccd-pcjfq" event={"ID":"4f352e98-8159-4205-9883-c4f945fe9e94","Type":"ContainerStarted","Data":"c83073979e172cc1d9fac9970e4e731e435f939205b6487ffb62055f6cc77d18"} Jul 28 17:09:43.772303 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:43.772282 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-859f88d764-48mxj" Jul 28 17:09:43.773995 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:43.773974 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n776k" event={"ID":"39aa1164-b895-45e9-b64a-5220376fdd95","Type":"ContainerStarted","Data":"bd70d7116ae59f567b43547aa98aa1f4ed1067488d11971bea5a0d5680d3e6d7"} Jul 28 17:09:43.793937 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:43.793890 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-859f88d764-48mxj" podStartSLOduration=7.711406607 podStartE2EDuration="13.793875362s" totalImagesPullingTime="6.082468755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:09:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:43.793053094 +0000 UTC m=+82.401984465" watchObservedRunningTime="2026-07-28 17:09:43.793875362 +0000 UTC m=+82.402806732" Jul 28 17:09:43.841190 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:43.841140 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b6bb8dccd-pcjfq" podStartSLOduration=7.929713057 podStartE2EDuration="13.84112357s" totalImagesPullingTime="5.911410513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:09:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:43.839808041 +0000 UTC m=+82.448739411" watchObservedRunningTime="2026-07-28 17:09:43.84112357 +0000 UTC m=+82.450054936" Jul 28 17:09:43.868300 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:43.868242 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-n776k" podStartSLOduration=17.080241164 podStartE2EDuration="37.868223696s" totalImagesPullingTime="20.787982532s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:09:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:43.866443254 +0000 UTC m=+82.475374626" watchObservedRunningTime="2026-07-28 17:09:43.868223696 +0000 UTC m=+82.477155087" Jul 28 17:09:48.756492 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:48.756387 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-49xwx" Jul 28 17:09:51.181001 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:51.180961 2570 patch_prober.go:28] interesting pod/image-registry-7b5446bd95-ngjw4 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 28 17:09:51.181416 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:51.181014 2570 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7b5446bd95-ngjw4" podUID="0aa92363-6609-459f-9ce0-bbc1700a8bd8" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:09:51.630242 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:51.630214 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" Jul 28 17:09:52.651396 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:52.651358 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-q7fs6"] Jul 28 17:09:52.656005 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:52.655985 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-q7fs6" Jul 28 17:09:52.659976 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:52.659955 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 28 17:09:52.660239 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:52.660223 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 28 17:09:52.660923 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:52.660907 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 28 17:09:52.660923 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:52.660918 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5mp2x\"" Jul 28 17:09:52.661503 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:52.661485 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 28 17:09:52.712283 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:52.712255 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f351cfd4-5d9a-4977-99fc-c37be3098628-sys\") pod \"node-exporter-q7fs6\" (UID: \"f351cfd4-5d9a-4977-99fc-c37be3098628\") " pod="openshift-monitoring/node-exporter-q7fs6" Jul 28 17:09:52.712435 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:52.712297 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f351cfd4-5d9a-4977-99fc-c37be3098628-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q7fs6\" (UID: \"f351cfd4-5d9a-4977-99fc-c37be3098628\") " pod="openshift-monitoring/node-exporter-q7fs6" Jul 28 17:09:52.712435 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:52.712326 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f351cfd4-5d9a-4977-99fc-c37be3098628-node-exporter-accelerators-collector-config\") pod \"node-exporter-q7fs6\" (UID: \"f351cfd4-5d9a-4977-99fc-c37be3098628\") " pod="openshift-monitoring/node-exporter-q7fs6" Jul 28 17:09:52.712435 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:52.712350 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f351cfd4-5d9a-4977-99fc-c37be3098628-node-exporter-tls\") pod \"node-exporter-q7fs6\" (UID: \"f351cfd4-5d9a-4977-99fc-c37be3098628\") " pod="openshift-monitoring/node-exporter-q7fs6" Jul 28 17:09:52.712546 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:52.712440 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f351cfd4-5d9a-4977-99fc-c37be3098628-node-exporter-wtmp\") pod \"node-exporter-q7fs6\" (UID: \"f351cfd4-5d9a-4977-99fc-c37be3098628\") " pod="openshift-monitoring/node-exporter-q7fs6" Jul 28 17:09:52.712546 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:52.712476 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f351cfd4-5d9a-4977-99fc-c37be3098628-metrics-client-ca\") pod \"node-exporter-q7fs6\" (UID: \"f351cfd4-5d9a-4977-99fc-c37be3098628\") " pod="openshift-monitoring/node-exporter-q7fs6" Jul 28 17:09:52.712546 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:52.712502 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t4ng\" (UniqueName: \"kubernetes.io/projected/f351cfd4-5d9a-4977-99fc-c37be3098628-kube-api-access-6t4ng\") pod \"node-exporter-q7fs6\" (UID: \"f351cfd4-5d9a-4977-99fc-c37be3098628\") " pod="openshift-monitoring/node-exporter-q7fs6" Jul 28 17:09:52.712637 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:52.712558 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f351cfd4-5d9a-4977-99fc-c37be3098628-node-exporter-textfile\") pod \"node-exporter-q7fs6\" (UID: \"f351cfd4-5d9a-4977-99fc-c37be3098628\") " pod="openshift-monitoring/node-exporter-q7fs6" Jul 28 17:09:52.712637 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:52.712586 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f351cfd4-5d9a-4977-99fc-c37be3098628-root\") pod \"node-exporter-q7fs6\" (UID: \"f351cfd4-5d9a-4977-99fc-c37be3098628\") " pod="openshift-monitoring/node-exporter-q7fs6" Jul 28 17:09:52.813298 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:52.813266 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f351cfd4-5d9a-4977-99fc-c37be3098628-metrics-client-ca\") pod \"node-exporter-q7fs6\" (UID: \"f351cfd4-5d9a-4977-99fc-c37be3098628\") " pod="openshift-monitoring/node-exporter-q7fs6" Jul 28 17:09:52.813478 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:52.813303 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6t4ng\" (UniqueName: \"kubernetes.io/projected/f351cfd4-5d9a-4977-99fc-c37be3098628-kube-api-access-6t4ng\") pod \"node-exporter-q7fs6\" (UID: \"f351cfd4-5d9a-4977-99fc-c37be3098628\") " pod="openshift-monitoring/node-exporter-q7fs6" Jul 28 17:09:52.813478 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:52.813327 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f351cfd4-5d9a-4977-99fc-c37be3098628-node-exporter-textfile\") pod \"node-exporter-q7fs6\" (UID: \"f351cfd4-5d9a-4977-99fc-c37be3098628\") " pod="openshift-monitoring/node-exporter-q7fs6" Jul 28 17:09:52.813478 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:52.813349 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f351cfd4-5d9a-4977-99fc-c37be3098628-root\") pod \"node-exporter-q7fs6\" (UID: \"f351cfd4-5d9a-4977-99fc-c37be3098628\") " pod="openshift-monitoring/node-exporter-q7fs6" Jul 28 17:09:52.813478 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:52.813367 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f351cfd4-5d9a-4977-99fc-c37be3098628-sys\") pod \"node-exporter-q7fs6\" (UID: \"f351cfd4-5d9a-4977-99fc-c37be3098628\") " pod="openshift-monitoring/node-exporter-q7fs6" Jul 28 17:09:52.813478 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:52.813384 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f351cfd4-5d9a-4977-99fc-c37be3098628-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q7fs6\" (UID: \"f351cfd4-5d9a-4977-99fc-c37be3098628\") " pod="openshift-monitoring/node-exporter-q7fs6" Jul 28 17:09:52.813478 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:52.813428 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f351cfd4-5d9a-4977-99fc-c37be3098628-root\") pod \"node-exporter-q7fs6\" (UID: \"f351cfd4-5d9a-4977-99fc-c37be3098628\") " pod="openshift-monitoring/node-exporter-q7fs6" Jul 28 17:09:52.813478 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:52.813440 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f351cfd4-5d9a-4977-99fc-c37be3098628-sys\") pod \"node-exporter-q7fs6\" (UID: \"f351cfd4-5d9a-4977-99fc-c37be3098628\") " pod="openshift-monitoring/node-exporter-q7fs6" Jul 28 17:09:52.813830 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:52.813482 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f351cfd4-5d9a-4977-99fc-c37be3098628-node-exporter-accelerators-collector-config\") pod \"node-exporter-q7fs6\" (UID: \"f351cfd4-5d9a-4977-99fc-c37be3098628\") " pod="openshift-monitoring/node-exporter-q7fs6" Jul 28 17:09:52.813830 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:52.813512 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f351cfd4-5d9a-4977-99fc-c37be3098628-node-exporter-tls\") pod \"node-exporter-q7fs6\" (UID: \"f351cfd4-5d9a-4977-99fc-c37be3098628\") " pod="openshift-monitoring/node-exporter-q7fs6" Jul 28 17:09:52.813830 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:52.813573 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f351cfd4-5d9a-4977-99fc-c37be3098628-node-exporter-wtmp\") pod \"node-exporter-q7fs6\" (UID: \"f351cfd4-5d9a-4977-99fc-c37be3098628\") " pod="openshift-monitoring/node-exporter-q7fs6" Jul 28 17:09:52.813830 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:52.813723 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f351cfd4-5d9a-4977-99fc-c37be3098628-node-exporter-wtmp\") pod \"node-exporter-q7fs6\" (UID: \"f351cfd4-5d9a-4977-99fc-c37be3098628\") " pod="openshift-monitoring/node-exporter-q7fs6" Jul 28 17:09:52.813830 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:52.813734 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f351cfd4-5d9a-4977-99fc-c37be3098628-node-exporter-textfile\") pod \"node-exporter-q7fs6\" (UID: \"f351cfd4-5d9a-4977-99fc-c37be3098628\") " pod="openshift-monitoring/node-exporter-q7fs6" Jul 28 17:09:52.814066 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:52.813984 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f351cfd4-5d9a-4977-99fc-c37be3098628-metrics-client-ca\") pod \"node-exporter-q7fs6\" (UID: \"f351cfd4-5d9a-4977-99fc-c37be3098628\") " pod="openshift-monitoring/node-exporter-q7fs6" Jul 28 17:09:52.814066 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:52.814008 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f351cfd4-5d9a-4977-99fc-c37be3098628-node-exporter-accelerators-collector-config\") pod \"node-exporter-q7fs6\" (UID: \"f351cfd4-5d9a-4977-99fc-c37be3098628\") " pod="openshift-monitoring/node-exporter-q7fs6" Jul 28 17:09:52.815669 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:52.815649 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f351cfd4-5d9a-4977-99fc-c37be3098628-node-exporter-tls\") pod \"node-exporter-q7fs6\" (UID: \"f351cfd4-5d9a-4977-99fc-c37be3098628\") " pod="openshift-monitoring/node-exporter-q7fs6" Jul 28 17:09:52.815750 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:52.815724 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f351cfd4-5d9a-4977-99fc-c37be3098628-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q7fs6\" (UID: \"f351cfd4-5d9a-4977-99fc-c37be3098628\") " pod="openshift-monitoring/node-exporter-q7fs6" Jul 28 17:09:52.826652 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:52.826626 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t4ng\" (UniqueName: \"kubernetes.io/projected/f351cfd4-5d9a-4977-99fc-c37be3098628-kube-api-access-6t4ng\") pod \"node-exporter-q7fs6\" (UID: \"f351cfd4-5d9a-4977-99fc-c37be3098628\") " pod="openshift-monitoring/node-exporter-q7fs6" Jul 28 17:09:52.965445 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:52.965374 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-q7fs6" Jul 28 17:09:52.977541 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:09:52.977514 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf351cfd4_5d9a_4977_99fc_c37be3098628.slice/crio-42bfd0152b004cfc020246678ee4beeea088120bd7ecb8453d8cac0ff852f1d1 WatchSource:0}: Error finding container 42bfd0152b004cfc020246678ee4beeea088120bd7ecb8453d8cac0ff852f1d1: Status 404 returned error can't find the container with id 42bfd0152b004cfc020246678ee4beeea088120bd7ecb8453d8cac0ff852f1d1 Jul 28 17:09:53.813080 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:53.813042 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q7fs6" event={"ID":"f351cfd4-5d9a-4977-99fc-c37be3098628","Type":"ContainerStarted","Data":"42bfd0152b004cfc020246678ee4beeea088120bd7ecb8453d8cac0ff852f1d1"} Jul 28 17:09:54.817934 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:54.817903 2570 generic.go:356] "Generic (PLEG): container finished" podID="f351cfd4-5d9a-4977-99fc-c37be3098628" containerID="9f570a3899c4ce420155242c62f9a6ac4b5b25c26e96a4283c606963bd2c175d" exitCode=0 Jul 28 17:09:54.818327 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:54.817999 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q7fs6" event={"ID":"f351cfd4-5d9a-4977-99fc-c37be3098628","Type":"ContainerDied","Data":"9f570a3899c4ce420155242c62f9a6ac4b5b25c26e96a4283c606963bd2c175d"} Jul 28 17:09:55.823991 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:55.823955 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q7fs6" event={"ID":"f351cfd4-5d9a-4977-99fc-c37be3098628","Type":"ContainerStarted","Data":"0ad9ee85e2b11e46e3b81e757a8aa7de5c726f80e994aff80b3fad454bc9385a"} Jul 28 17:09:55.824407 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:55.824034 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q7fs6" event={"ID":"f351cfd4-5d9a-4977-99fc-c37be3098628","Type":"ContainerStarted","Data":"3faa65685f3aba49713186fc6c13161115448c35c86238031c3c69d54b363bd2"} Jul 28 17:09:56.644530 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:56.644468 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" podUID="2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c" containerName="registry" containerID="cri-o://6355a36781abcf2ed3e6e6b468b9cd89daaa0b4cdb5b59e06028fe0b3ee78fcb" gracePeriod=30 Jul 28 17:09:56.830156 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:56.830104 2570 generic.go:356] "Generic (PLEG): container finished" podID="2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c" containerID="6355a36781abcf2ed3e6e6b468b9cd89daaa0b4cdb5b59e06028fe0b3ee78fcb" exitCode=0 Jul 28 17:09:56.830549 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:56.830151 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" event={"ID":"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c","Type":"ContainerDied","Data":"6355a36781abcf2ed3e6e6b468b9cd89daaa0b4cdb5b59e06028fe0b3ee78fcb"} Jul 28 17:09:56.889629 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:56.889601 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" Jul 28 17:09:56.919229 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:56.919140 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-q7fs6" podStartSLOduration=3.852225924 podStartE2EDuration="4.919108442s" totalImagesPullingTime="1.002108735s" totalInitContainerRuntime="64.773783ms" isStatefulPod=true podCreationTimestamp="2026-07-28 17:09:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:55.855002409 +0000 UTC m=+94.463933780" watchObservedRunningTime="2026-07-28 17:09:56.919108442 +0000 UTC m=+95.528039864" Jul 28 17:09:56.953507 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:56.953477 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-trusted-ca\") pod \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\" (UID: \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\") " Jul 28 17:09:56.953638 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:56.953518 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-registry-tls\") pod \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\" (UID: \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\") " Jul 28 17:09:56.953638 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:56.953543 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-bound-sa-token\") pod \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\" (UID: \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\") " Jul 28 17:09:56.953638 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:56.953565 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-glntn\" (UniqueName: \"kubernetes.io/projected/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-kube-api-access-glntn\") pod \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\" (UID: \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\") " Jul 28 17:09:56.953638 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:56.953597 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-registry-certificates\") pod \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\" (UID: \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\") " Jul 28 17:09:56.953806 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:56.953639 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-installation-pull-secrets\") pod \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\" (UID: \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\") " Jul 28 17:09:56.953806 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:56.953690 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-ca-trust-extracted\") pod \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\" (UID: \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\") " Jul 28 17:09:56.953806 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:56.953716 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-image-registry-private-configuration\") pod \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\" (UID: \"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c\") " Jul 28 17:09:56.955786 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:56.953980 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c" (UID: "2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:09:56.955786 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:56.954059 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c" (UID: "2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:09:56.955786 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:56.954072 2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-trusted-ca\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:09:56.956225 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:56.956178 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c" (UID: "2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:09:56.956309 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:56.956216 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c" (UID: "2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:09:56.956403 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:56.956381 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-kube-api-access-glntn" (OuterVolumeSpecName: "kube-api-access-glntn") pod "2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c" (UID: "2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c"). InnerVolumeSpecName "kube-api-access-glntn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:09:56.956447 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:56.956432 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c" (UID: "2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:09:56.956748 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:56.956726 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c" (UID: "2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:09:56.963090 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:56.963065 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c" (UID: "2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:09:57.055032 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:57.054995 2570 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-registry-certificates\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:09:57.055032 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:57.055024 2570 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-installation-pull-secrets\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:09:57.055032 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:57.055038 2570 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-ca-trust-extracted\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:09:57.055281 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:57.055053 2570 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-image-registry-private-configuration\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:09:57.055281 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:57.055066 2570 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-registry-tls\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:09:57.055281 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:57.055078 2570 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-bound-sa-token\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:09:57.055281 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:57.055088 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-glntn\" (UniqueName: \"kubernetes.io/projected/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c-kube-api-access-glntn\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:09:57.835588 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:57.835556 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" event={"ID":"2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c","Type":"ContainerDied","Data":"76628011d9cee26fb33c5a8f14a2234906931680ca7fff0be9e593e9d8b9b649"} Jul 28 17:09:57.835967 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:57.835612 2570 scope.go:117] "RemoveContainer" containerID="6355a36781abcf2ed3e6e6b468b9cd89daaa0b4cdb5b59e06028fe0b3ee78fcb" Jul 28 17:09:57.835967 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:57.835569 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b5f57f94b-vgwb7" Jul 28 17:09:57.865266 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:57.865234 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6b5f57f94b-vgwb7"] Jul 28 17:09:57.874591 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:57.874569 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6b5f57f94b-vgwb7"] Jul 28 17:09:58.119031 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:58.118963 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c" path="/var/lib/kubelet/pods/2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c/volumes" Jul 28 17:09:59.616830 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:09:59.616803 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-6dbsb" Jul 28 17:10:00.760958 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:10:00.760928 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7b5446bd95-ngjw4" Jul 28 17:10:21.922894 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:10:21.922859 2570 generic.go:356] "Generic (PLEG): container finished" podID="9a22bd3d-e2c0-461e-b364-b02d99e87eb9" containerID="10e403ba03868c5ac447e7fc342b40fbade6d4009ed6598050f243c8fc7053c9" exitCode=0 Jul 28 17:10:21.923253 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:10:21.922930 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-tl7rc" event={"ID":"9a22bd3d-e2c0-461e-b364-b02d99e87eb9","Type":"ContainerDied","Data":"10e403ba03868c5ac447e7fc342b40fbade6d4009ed6598050f243c8fc7053c9"} Jul 28 17:10:21.923294 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:10:21.923261 2570 scope.go:117] "RemoveContainer" containerID="10e403ba03868c5ac447e7fc342b40fbade6d4009ed6598050f243c8fc7053c9" Jul 28 17:10:22.928195 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:10:22.928156 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-tl7rc" event={"ID":"9a22bd3d-e2c0-461e-b364-b02d99e87eb9","Type":"ContainerStarted","Data":"98e5f02296daf09466086514865233e5987f22ab201a8ba6d5ee92c19accac9a"} Jul 28 17:10:26.943308 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:10:26.943273 2570 generic.go:356] "Generic (PLEG): container finished" podID="4ddc4d25-529e-44f1-a922-1516040c72f4" containerID="5085a92bdd988494ceba8591e52367cf0a8f7d262b5d08b1d9a95f6f5c5c5a57" exitCode=0 Jul 28 17:10:26.943738 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:10:26.943324 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-vznd7" event={"ID":"4ddc4d25-529e-44f1-a922-1516040c72f4","Type":"ContainerDied","Data":"5085a92bdd988494ceba8591e52367cf0a8f7d262b5d08b1d9a95f6f5c5c5a57"} Jul 28 17:10:26.943738 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:10:26.943624 2570 scope.go:117] "RemoveContainer" containerID="5085a92bdd988494ceba8591e52367cf0a8f7d262b5d08b1d9a95f6f5c5c5a57" Jul 28 17:10:27.948217 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:10:27.948182 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-vznd7" event={"ID":"4ddc4d25-529e-44f1-a922-1516040c72f4","Type":"ContainerStarted","Data":"d5e7936f312a4608df6bcbd6eefad714684f04bc2cc8dc811aa5a763fb991e30"} Jul 28 17:10:36.982955 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:10:36.982911 2570 generic.go:356] "Generic (PLEG): container finished" podID="8f9b8ad8-774d-4ab8-9826-94eb5a37743f" containerID="b1a154d20b8f9888cf5dc3364314d24277362efbc6a5c3b400b41d002faa49ce" exitCode=0 Jul 28 17:10:36.983363 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:10:36.982981 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-bpkxs" event={"ID":"8f9b8ad8-774d-4ab8-9826-94eb5a37743f","Type":"ContainerDied","Data":"b1a154d20b8f9888cf5dc3364314d24277362efbc6a5c3b400b41d002faa49ce"} Jul 28 17:10:36.983363 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:10:36.983354 2570 scope.go:117] "RemoveContainer" containerID="b1a154d20b8f9888cf5dc3364314d24277362efbc6a5c3b400b41d002faa49ce" Jul 28 17:10:37.988560 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:10:37.988530 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-bpkxs" event={"ID":"8f9b8ad8-774d-4ab8-9826-94eb5a37743f","Type":"ContainerStarted","Data":"7d22004c8dfc8707861757d8187f85edac02ed131aef97bc1f01ea68f2ffb9b5"} Jul 28 17:12:54.864661 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:12:54.864581 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rvktb"] Jul 28 17:12:54.865094 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:12:54.864891 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c" containerName="registry" Jul 28 17:12:54.865094 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:12:54.864905 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c" containerName="registry" Jul 28 17:12:54.865094 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:12:54.864971 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d8be12f-23ea-4ddf-bc6d-9bb0cf9da50c" containerName="registry" Jul 28 17:12:54.867707 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:12:54.867692 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rvktb" Jul 28 17:12:54.870613 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:12:54.870585 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-f68vv\"" Jul 28 17:12:54.870613 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:12:54.870597 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 28 17:12:54.870774 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:12:54.870665 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 28 17:12:54.892577 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:12:54.892548 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rvktb"] Jul 28 17:12:54.960219 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:12:54.960185 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/584eaf70-7b39-4e99-81b9-5221dd61f137-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rvktb\" (UID: \"584eaf70-7b39-4e99-81b9-5221dd61f137\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rvktb" Jul 28 17:12:54.960388 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:12:54.960229 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdj8s\" (UniqueName: \"kubernetes.io/projected/584eaf70-7b39-4e99-81b9-5221dd61f137-kube-api-access-jdj8s\") pod \"cert-manager-operator-controller-manager-55c444bf86-rvktb\" (UID: \"584eaf70-7b39-4e99-81b9-5221dd61f137\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rvktb" Jul 28 17:12:55.060926 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:12:55.060888 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/584eaf70-7b39-4e99-81b9-5221dd61f137-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rvktb\" (UID: \"584eaf70-7b39-4e99-81b9-5221dd61f137\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rvktb" Jul 28 17:12:55.060926 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:12:55.060932 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdj8s\" (UniqueName: \"kubernetes.io/projected/584eaf70-7b39-4e99-81b9-5221dd61f137-kube-api-access-jdj8s\") pod \"cert-manager-operator-controller-manager-55c444bf86-rvktb\" (UID: \"584eaf70-7b39-4e99-81b9-5221dd61f137\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rvktb" Jul 28 17:12:55.061279 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:12:55.061261 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/584eaf70-7b39-4e99-81b9-5221dd61f137-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rvktb\" (UID: \"584eaf70-7b39-4e99-81b9-5221dd61f137\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rvktb" Jul 28 17:12:55.070798 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:12:55.070768 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdj8s\" (UniqueName: \"kubernetes.io/projected/584eaf70-7b39-4e99-81b9-5221dd61f137-kube-api-access-jdj8s\") pod \"cert-manager-operator-controller-manager-55c444bf86-rvktb\" (UID: \"584eaf70-7b39-4e99-81b9-5221dd61f137\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rvktb" Jul 28 17:12:55.176384 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:12:55.176345 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rvktb" Jul 28 17:12:55.317933 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:12:55.317875 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rvktb"] Jul 28 17:12:55.323678 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:12:55.323641 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod584eaf70_7b39_4e99_81b9_5221dd61f137.slice/crio-902589ee761de18887fbf7dd346cd978059b111b39d0a3063cb69a13117a71bb WatchSource:0}: Error finding container 902589ee761de18887fbf7dd346cd978059b111b39d0a3063cb69a13117a71bb: Status 404 returned error can't find the container with id 902589ee761de18887fbf7dd346cd978059b111b39d0a3063cb69a13117a71bb Jul 28 17:12:55.488634 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:12:55.488552 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rvktb" event={"ID":"584eaf70-7b39-4e99-81b9-5221dd61f137","Type":"ContainerStarted","Data":"902589ee761de18887fbf7dd346cd978059b111b39d0a3063cb69a13117a71bb"} Jul 28 17:12:58.502786 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:12:58.502750 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rvktb" event={"ID":"584eaf70-7b39-4e99-81b9-5221dd61f137","Type":"ContainerStarted","Data":"083a1e9f00fa89a1717e1a2939a8e74d511f2c91bf80921dbaff288a20a95bd4"} Jul 28 17:12:58.555261 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:12:58.555205 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rvktb" podStartSLOduration=2.072488171 podStartE2EDuration="4.555189871s" totalImagesPullingTime="2.4827017s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:12:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:12:58.553891546 +0000 UTC m=+277.162822917" watchObservedRunningTime="2026-07-28 17:12:58.555189871 +0000 UTC m=+277.164121240" Jul 28 17:13:21.803348 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:13:21.803314 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-slfqf_ecb82800-8cf0-4ef9-926c-155172fad7ff/console-operator/1.log" Jul 28 17:13:21.812363 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:13:21.812338 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-slfqf_ecb82800-8cf0-4ef9-926c-155172fad7ff/console-operator/1.log" Jul 28 17:13:21.814633 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:13:21.814611 2570 kubelet.go:1648] "Image garbage collection succeeded" Jul 28 17:13:44.182568 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:13:44.182536 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d8c7b7dcd-lc72m"] Jul 28 17:13:44.185891 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:13:44.185874 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d8c7b7dcd-lc72m" Jul 28 17:13:44.190901 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:13:44.190882 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 28 17:13:44.191026 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:13:44.190884 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-m9txd\"" Jul 28 17:13:44.191273 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:13:44.191254 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 28 17:13:44.191505 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:13:44.191491 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 28 17:13:44.191562 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:13:44.191534 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 28 17:13:44.206514 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:13:44.206487 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d8c7b7dcd-lc72m"] Jul 28 17:13:44.346720 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:13:44.346684 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4c258043-1b2d-4bd2-abfb-c46a681d788b-webhook-cert\") pod \"opendatahub-operator-controller-manager-d8c7b7dcd-lc72m\" (UID: \"4c258043-1b2d-4bd2-abfb-c46a681d788b\") " pod="opendatahub/opendatahub-operator-controller-manager-d8c7b7dcd-lc72m" Jul 28 17:13:44.346720 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:13:44.346718 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwd2l\" (UniqueName: \"kubernetes.io/projected/4c258043-1b2d-4bd2-abfb-c46a681d788b-kube-api-access-bwd2l\") pod \"opendatahub-operator-controller-manager-d8c7b7dcd-lc72m\" (UID: \"4c258043-1b2d-4bd2-abfb-c46a681d788b\") " pod="opendatahub/opendatahub-operator-controller-manager-d8c7b7dcd-lc72m" Jul 28 17:13:44.346923 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:13:44.346743 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4c258043-1b2d-4bd2-abfb-c46a681d788b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d8c7b7dcd-lc72m\" (UID: \"4c258043-1b2d-4bd2-abfb-c46a681d788b\") " pod="opendatahub/opendatahub-operator-controller-manager-d8c7b7dcd-lc72m" Jul 28 17:13:44.346923 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:13:44.346766 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c258043-1b2d-4bd2-abfb-c46a681d788b-tmp\") pod \"opendatahub-operator-controller-manager-d8c7b7dcd-lc72m\" (UID: \"4c258043-1b2d-4bd2-abfb-c46a681d788b\") " pod="opendatahub/opendatahub-operator-controller-manager-d8c7b7dcd-lc72m" Jul 28 17:13:44.346923 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:13:44.346823 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4c258043-1b2d-4bd2-abfb-c46a681d788b-manifests\") pod \"opendatahub-operator-controller-manager-d8c7b7dcd-lc72m\" (UID: \"4c258043-1b2d-4bd2-abfb-c46a681d788b\") " pod="opendatahub/opendatahub-operator-controller-manager-d8c7b7dcd-lc72m" Jul 28 17:13:44.447261 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:13:44.447187 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4c258043-1b2d-4bd2-abfb-c46a681d788b-webhook-cert\") pod \"opendatahub-operator-controller-manager-d8c7b7dcd-lc72m\" (UID: \"4c258043-1b2d-4bd2-abfb-c46a681d788b\") " pod="opendatahub/opendatahub-operator-controller-manager-d8c7b7dcd-lc72m" Jul 28 17:13:44.447261 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:13:44.447224 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwd2l\" (UniqueName: \"kubernetes.io/projected/4c258043-1b2d-4bd2-abfb-c46a681d788b-kube-api-access-bwd2l\") pod \"opendatahub-operator-controller-manager-d8c7b7dcd-lc72m\" (UID: \"4c258043-1b2d-4bd2-abfb-c46a681d788b\") " pod="opendatahub/opendatahub-operator-controller-manager-d8c7b7dcd-lc72m" Jul 28 17:13:44.447261 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:13:44.447246 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4c258043-1b2d-4bd2-abfb-c46a681d788b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d8c7b7dcd-lc72m\" (UID: \"4c258043-1b2d-4bd2-abfb-c46a681d788b\") " pod="opendatahub/opendatahub-operator-controller-manager-d8c7b7dcd-lc72m" Jul 28 17:13:44.447513 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:13:44.447273 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c258043-1b2d-4bd2-abfb-c46a681d788b-tmp\") pod \"opendatahub-operator-controller-manager-d8c7b7dcd-lc72m\" (UID: \"4c258043-1b2d-4bd2-abfb-c46a681d788b\") " pod="opendatahub/opendatahub-operator-controller-manager-d8c7b7dcd-lc72m" Jul 28 17:13:44.447513 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:13:44.447303 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4c258043-1b2d-4bd2-abfb-c46a681d788b-manifests\") pod \"opendatahub-operator-controller-manager-d8c7b7dcd-lc72m\" (UID: \"4c258043-1b2d-4bd2-abfb-c46a681d788b\") " pod="opendatahub/opendatahub-operator-controller-manager-d8c7b7dcd-lc72m" Jul 28 17:13:44.447647 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:13:44.447629 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4c258043-1b2d-4bd2-abfb-c46a681d788b-manifests\") pod \"opendatahub-operator-controller-manager-d8c7b7dcd-lc72m\" (UID: \"4c258043-1b2d-4bd2-abfb-c46a681d788b\") " pod="opendatahub/opendatahub-operator-controller-manager-d8c7b7dcd-lc72m" Jul 28 17:13:44.449460 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:13:44.449430 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c258043-1b2d-4bd2-abfb-c46a681d788b-tmp\") pod \"opendatahub-operator-controller-manager-d8c7b7dcd-lc72m\" (UID: \"4c258043-1b2d-4bd2-abfb-c46a681d788b\") " pod="opendatahub/opendatahub-operator-controller-manager-d8c7b7dcd-lc72m" Jul 28 17:13:44.449579 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:13:44.449530 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4c258043-1b2d-4bd2-abfb-c46a681d788b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d8c7b7dcd-lc72m\" (UID: \"4c258043-1b2d-4bd2-abfb-c46a681d788b\") " pod="opendatahub/opendatahub-operator-controller-manager-d8c7b7dcd-lc72m" Jul 28 17:13:44.449579 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:13:44.449556 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4c258043-1b2d-4bd2-abfb-c46a681d788b-webhook-cert\") pod \"opendatahub-operator-controller-manager-d8c7b7dcd-lc72m\" (UID: \"4c258043-1b2d-4bd2-abfb-c46a681d788b\") " pod="opendatahub/opendatahub-operator-controller-manager-d8c7b7dcd-lc72m" Jul 28 17:13:44.457815 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:13:44.457793 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwd2l\" (UniqueName: \"kubernetes.io/projected/4c258043-1b2d-4bd2-abfb-c46a681d788b-kube-api-access-bwd2l\") pod \"opendatahub-operator-controller-manager-d8c7b7dcd-lc72m\" (UID: \"4c258043-1b2d-4bd2-abfb-c46a681d788b\") " pod="opendatahub/opendatahub-operator-controller-manager-d8c7b7dcd-lc72m" Jul 28 17:13:44.495753 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:13:44.495714 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d8c7b7dcd-lc72m" Jul 28 17:13:44.632681 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:13:44.632654 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d8c7b7dcd-lc72m"] Jul 28 17:13:44.635501 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:13:44.635473 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c258043_1b2d_4bd2_abfb_c46a681d788b.slice/crio-8299470c393505a4d798946bf1c63420d4d49598ff1a325ff19b10d2f5a4bc1c WatchSource:0}: Error finding container 8299470c393505a4d798946bf1c63420d4d49598ff1a325ff19b10d2f5a4bc1c: Status 404 returned error can't find the container with id 8299470c393505a4d798946bf1c63420d4d49598ff1a325ff19b10d2f5a4bc1c Jul 28 17:13:44.637229 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:13:44.637211 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 17:13:44.680620 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:13:44.680580 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d8c7b7dcd-lc72m" event={"ID":"4c258043-1b2d-4bd2-abfb-c46a681d788b","Type":"ContainerStarted","Data":"8299470c393505a4d798946bf1c63420d4d49598ff1a325ff19b10d2f5a4bc1c"} Jul 28 17:13:48.701819 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:13:48.701781 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d8c7b7dcd-lc72m" event={"ID":"4c258043-1b2d-4bd2-abfb-c46a681d788b","Type":"ContainerStarted","Data":"7f6043cd663de338c547236a44aa994b54f1655f7fd5c8df538ca1f31d8592f9"} Jul 28 17:13:49.707783 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:13:49.707749 2570 generic.go:356] "Generic (PLEG): container finished" podID="4c258043-1b2d-4bd2-abfb-c46a681d788b" containerID="7f6043cd663de338c547236a44aa994b54f1655f7fd5c8df538ca1f31d8592f9" exitCode=0 Jul 28 17:13:49.708182 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:13:49.707833 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d8c7b7dcd-lc72m" event={"ID":"4c258043-1b2d-4bd2-abfb-c46a681d788b","Type":"ContainerDied","Data":"7f6043cd663de338c547236a44aa994b54f1655f7fd5c8df538ca1f31d8592f9"} Jul 28 17:13:50.714071 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:13:50.714034 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d8c7b7dcd-lc72m" event={"ID":"4c258043-1b2d-4bd2-abfb-c46a681d788b","Type":"ContainerStarted","Data":"92ed3d99b481757ff40c99776912e40dbe8be6cdd4331f22169e7b6b184d95ea"} Jul 28 17:13:50.714556 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:13:50.714157 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-d8c7b7dcd-lc72m" Jul 28 17:13:50.753878 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:13:50.753821 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-d8c7b7dcd-lc72m" podStartSLOduration=2.052973093 podStartE2EDuration="6.753804739s" totalImagesPullingTime="3.615705648s" totalInitContainerRuntime="1.085125998s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:13:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:13:50.752008505 +0000 UTC m=+329.360939893" watchObservedRunningTime="2026-07-28 17:13:50.753804739 +0000 UTC m=+329.362736108" Jul 28 17:14:01.721160 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:01.721107 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-d8c7b7dcd-lc72m" Jul 28 17:14:04.466186 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:04.466147 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-77b678cfbc-gg5gz"] Jul 28 17:14:04.477095 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:04.477070 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-77b678cfbc-gg5gz" Jul 28 17:14:04.480274 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:04.480254 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 28 17:14:04.481814 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:04.481788 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 28 17:14:04.483010 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:04.482987 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-d5qnv\"" Jul 28 17:14:04.483520 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:04.483498 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-77b678cfbc-gg5gz"] Jul 28 17:14:04.503094 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:04.503065 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x576n\" (UniqueName: \"kubernetes.io/projected/1d224fae-cf17-4bdb-baf5-1dde1cd43871-kube-api-access-x576n\") pod \"kube-auth-proxy-77b678cfbc-gg5gz\" (UID: \"1d224fae-cf17-4bdb-baf5-1dde1cd43871\") " pod="openshift-ingress/kube-auth-proxy-77b678cfbc-gg5gz" Jul 28 17:14:04.503253 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:04.503150 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d224fae-cf17-4bdb-baf5-1dde1cd43871-tmp\") pod \"kube-auth-proxy-77b678cfbc-gg5gz\" (UID: \"1d224fae-cf17-4bdb-baf5-1dde1cd43871\") " pod="openshift-ingress/kube-auth-proxy-77b678cfbc-gg5gz" Jul 28 17:14:04.503253 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:04.503195 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d224fae-cf17-4bdb-baf5-1dde1cd43871-tls-certs\") pod \"kube-auth-proxy-77b678cfbc-gg5gz\" (UID: \"1d224fae-cf17-4bdb-baf5-1dde1cd43871\") " pod="openshift-ingress/kube-auth-proxy-77b678cfbc-gg5gz" Jul 28 17:14:04.603922 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:04.603885 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x576n\" (UniqueName: \"kubernetes.io/projected/1d224fae-cf17-4bdb-baf5-1dde1cd43871-kube-api-access-x576n\") pod \"kube-auth-proxy-77b678cfbc-gg5gz\" (UID: \"1d224fae-cf17-4bdb-baf5-1dde1cd43871\") " pod="openshift-ingress/kube-auth-proxy-77b678cfbc-gg5gz" Jul 28 17:14:04.604173 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:04.603943 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d224fae-cf17-4bdb-baf5-1dde1cd43871-tmp\") pod \"kube-auth-proxy-77b678cfbc-gg5gz\" (UID: \"1d224fae-cf17-4bdb-baf5-1dde1cd43871\") " pod="openshift-ingress/kube-auth-proxy-77b678cfbc-gg5gz" Jul 28 17:14:04.604173 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:04.603987 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d224fae-cf17-4bdb-baf5-1dde1cd43871-tls-certs\") pod \"kube-auth-proxy-77b678cfbc-gg5gz\" (UID: \"1d224fae-cf17-4bdb-baf5-1dde1cd43871\") " pod="openshift-ingress/kube-auth-proxy-77b678cfbc-gg5gz" Jul 28 17:14:04.606229 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:04.606206 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d224fae-cf17-4bdb-baf5-1dde1cd43871-tmp\") pod \"kube-auth-proxy-77b678cfbc-gg5gz\" (UID: \"1d224fae-cf17-4bdb-baf5-1dde1cd43871\") " pod="openshift-ingress/kube-auth-proxy-77b678cfbc-gg5gz" Jul 28 17:14:04.606441 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:04.606423 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d224fae-cf17-4bdb-baf5-1dde1cd43871-tls-certs\") pod \"kube-auth-proxy-77b678cfbc-gg5gz\" (UID: \"1d224fae-cf17-4bdb-baf5-1dde1cd43871\") " pod="openshift-ingress/kube-auth-proxy-77b678cfbc-gg5gz" Jul 28 17:14:04.616962 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:04.616939 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x576n\" (UniqueName: \"kubernetes.io/projected/1d224fae-cf17-4bdb-baf5-1dde1cd43871-kube-api-access-x576n\") pod \"kube-auth-proxy-77b678cfbc-gg5gz\" (UID: \"1d224fae-cf17-4bdb-baf5-1dde1cd43871\") " pod="openshift-ingress/kube-auth-proxy-77b678cfbc-gg5gz" Jul 28 17:14:04.633872 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:04.633842 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-mpxpz"] Jul 28 17:14:04.637176 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:04.637155 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-mpxpz" Jul 28 17:14:04.641420 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:04.641391 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 28 17:14:04.641420 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:04.641415 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-h9mk6\"" Jul 28 17:14:04.649053 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:04.649029 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-mpxpz"] Jul 28 17:14:04.704867 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:04.704832 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qf7h\" (UniqueName: \"kubernetes.io/projected/650f3a82-4de5-4bd2-ba3b-4f9c4c666f53-kube-api-access-2qf7h\") pod \"odh-model-controller-7998c67dc4-mpxpz\" (UID: \"650f3a82-4de5-4bd2-ba3b-4f9c4c666f53\") " pod="opendatahub/odh-model-controller-7998c67dc4-mpxpz" Jul 28 17:14:04.705019 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:04.704873 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/650f3a82-4de5-4bd2-ba3b-4f9c4c666f53-cert\") pod \"odh-model-controller-7998c67dc4-mpxpz\" (UID: \"650f3a82-4de5-4bd2-ba3b-4f9c4c666f53\") " pod="opendatahub/odh-model-controller-7998c67dc4-mpxpz" Jul 28 17:14:04.788244 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:04.788158 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-77b678cfbc-gg5gz" Jul 28 17:14:04.806186 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:04.806154 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qf7h\" (UniqueName: \"kubernetes.io/projected/650f3a82-4de5-4bd2-ba3b-4f9c4c666f53-kube-api-access-2qf7h\") pod \"odh-model-controller-7998c67dc4-mpxpz\" (UID: \"650f3a82-4de5-4bd2-ba3b-4f9c4c666f53\") " pod="opendatahub/odh-model-controller-7998c67dc4-mpxpz" Jul 28 17:14:04.806316 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:04.806207 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/650f3a82-4de5-4bd2-ba3b-4f9c4c666f53-cert\") pod \"odh-model-controller-7998c67dc4-mpxpz\" (UID: \"650f3a82-4de5-4bd2-ba3b-4f9c4c666f53\") " pod="opendatahub/odh-model-controller-7998c67dc4-mpxpz" Jul 28 17:14:04.806383 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:14:04.806363 2570 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 28 17:14:04.806445 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:14:04.806435 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/650f3a82-4de5-4bd2-ba3b-4f9c4c666f53-cert podName:650f3a82-4de5-4bd2-ba3b-4f9c4c666f53 nodeName:}" failed. No retries permitted until 2026-07-28 17:14:05.306419717 +0000 UTC m=+343.915351065 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/650f3a82-4de5-4bd2-ba3b-4f9c4c666f53-cert") pod "odh-model-controller-7998c67dc4-mpxpz" (UID: "650f3a82-4de5-4bd2-ba3b-4f9c4c666f53") : secret "odh-model-controller-webhook-cert" not found Jul 28 17:14:04.817590 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:04.817560 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qf7h\" (UniqueName: \"kubernetes.io/projected/650f3a82-4de5-4bd2-ba3b-4f9c4c666f53-kube-api-access-2qf7h\") pod \"odh-model-controller-7998c67dc4-mpxpz\" (UID: \"650f3a82-4de5-4bd2-ba3b-4f9c4c666f53\") " pod="opendatahub/odh-model-controller-7998c67dc4-mpxpz" Jul 28 17:14:04.932998 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:04.932959 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-77b678cfbc-gg5gz"] Jul 28 17:14:04.935101 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:14:04.935071 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d224fae_cf17_4bdb_baf5_1dde1cd43871.slice/crio-131de6e2060b737f261a3177af77fe3ab5419014f978fc4fb5fd1749573d1789 WatchSource:0}: Error finding container 131de6e2060b737f261a3177af77fe3ab5419014f978fc4fb5fd1749573d1789: Status 404 returned error can't find the container with id 131de6e2060b737f261a3177af77fe3ab5419014f978fc4fb5fd1749573d1789 Jul 28 17:14:05.310618 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:05.310583 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/650f3a82-4de5-4bd2-ba3b-4f9c4c666f53-cert\") pod \"odh-model-controller-7998c67dc4-mpxpz\" (UID: \"650f3a82-4de5-4bd2-ba3b-4f9c4c666f53\") " pod="opendatahub/odh-model-controller-7998c67dc4-mpxpz" Jul 28 17:14:05.312920 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:05.312900 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/650f3a82-4de5-4bd2-ba3b-4f9c4c666f53-cert\") pod \"odh-model-controller-7998c67dc4-mpxpz\" (UID: \"650f3a82-4de5-4bd2-ba3b-4f9c4c666f53\") " pod="opendatahub/odh-model-controller-7998c67dc4-mpxpz" Jul 28 17:14:05.547754 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:05.547716 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-mpxpz" Jul 28 17:14:05.709504 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:05.709473 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-mpxpz"] Jul 28 17:14:05.712055 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:14:05.712011 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod650f3a82_4de5_4bd2_ba3b_4f9c4c666f53.slice/crio-16aa8adf5f8e8c53391af69f3fff2e07a080ff260af716be4146f8f57f00976d WatchSource:0}: Error finding container 16aa8adf5f8e8c53391af69f3fff2e07a080ff260af716be4146f8f57f00976d: Status 404 returned error can't find the container with id 16aa8adf5f8e8c53391af69f3fff2e07a080ff260af716be4146f8f57f00976d Jul 28 17:14:05.775651 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:05.775614 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-mpxpz" event={"ID":"650f3a82-4de5-4bd2-ba3b-4f9c4c666f53","Type":"ContainerStarted","Data":"16aa8adf5f8e8c53391af69f3fff2e07a080ff260af716be4146f8f57f00976d"} Jul 28 17:14:05.776883 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:05.776857 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-77b678cfbc-gg5gz" event={"ID":"1d224fae-cf17-4bdb-baf5-1dde1cd43871","Type":"ContainerStarted","Data":"131de6e2060b737f261a3177af77fe3ab5419014f978fc4fb5fd1749573d1789"} Jul 28 17:14:05.811219 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:05.811188 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-7t86x"] Jul 28 17:14:05.814628 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:05.814607 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-7t86x" Jul 28 17:14:05.820443 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:05.820421 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-k749q\"" Jul 28 17:14:05.829897 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:05.829877 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 28 17:14:05.839403 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:05.839376 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-7t86x"] Jul 28 17:14:05.914456 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:05.914427 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9eb4c97f-be9f-4584-b563-9a7e7efc1d2a-tls-certs\") pod \"model-serving-api-54d89bd5fc-7t86x\" (UID: \"9eb4c97f-be9f-4584-b563-9a7e7efc1d2a\") " pod="opendatahub/model-serving-api-54d89bd5fc-7t86x" Jul 28 17:14:05.914615 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:05.914486 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c52xp\" (UniqueName: \"kubernetes.io/projected/9eb4c97f-be9f-4584-b563-9a7e7efc1d2a-kube-api-access-c52xp\") pod \"model-serving-api-54d89bd5fc-7t86x\" (UID: \"9eb4c97f-be9f-4584-b563-9a7e7efc1d2a\") " pod="opendatahub/model-serving-api-54d89bd5fc-7t86x" Jul 28 17:14:06.014960 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:06.014920 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c52xp\" (UniqueName: \"kubernetes.io/projected/9eb4c97f-be9f-4584-b563-9a7e7efc1d2a-kube-api-access-c52xp\") pod \"model-serving-api-54d89bd5fc-7t86x\" (UID: \"9eb4c97f-be9f-4584-b563-9a7e7efc1d2a\") " pod="opendatahub/model-serving-api-54d89bd5fc-7t86x" Jul 28 17:14:06.015169 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:06.014992 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9eb4c97f-be9f-4584-b563-9a7e7efc1d2a-tls-certs\") pod \"model-serving-api-54d89bd5fc-7t86x\" (UID: \"9eb4c97f-be9f-4584-b563-9a7e7efc1d2a\") " pod="opendatahub/model-serving-api-54d89bd5fc-7t86x" Jul 28 17:14:06.017754 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:06.017721 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9eb4c97f-be9f-4584-b563-9a7e7efc1d2a-tls-certs\") pod \"model-serving-api-54d89bd5fc-7t86x\" (UID: \"9eb4c97f-be9f-4584-b563-9a7e7efc1d2a\") " pod="opendatahub/model-serving-api-54d89bd5fc-7t86x" Jul 28 17:14:06.033631 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:06.033598 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c52xp\" (UniqueName: \"kubernetes.io/projected/9eb4c97f-be9f-4584-b563-9a7e7efc1d2a-kube-api-access-c52xp\") pod \"model-serving-api-54d89bd5fc-7t86x\" (UID: \"9eb4c97f-be9f-4584-b563-9a7e7efc1d2a\") " pod="opendatahub/model-serving-api-54d89bd5fc-7t86x" Jul 28 17:14:06.126478 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:06.126444 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-7t86x" Jul 28 17:14:06.212161 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:06.212125 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d"] Jul 28 17:14:06.217048 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:06.216799 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d" Jul 28 17:14:06.220593 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:06.220573 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 28 17:14:06.221254 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:06.221225 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 28 17:14:06.221446 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:06.221430 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-7dpg4\"" Jul 28 17:14:06.221538 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:06.221472 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 28 17:14:06.221687 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:06.221674 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 28 17:14:06.238194 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:06.238161 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d"] Jul 28 17:14:06.293149 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:06.293106 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-7t86x"] Jul 28 17:14:06.293616 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:14:06.293582 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eb4c97f_be9f_4584_b563_9a7e7efc1d2a.slice/crio-167cb7147e4f08312a51e2018ac039e7057dd77d730f6955248dea999912318f WatchSource:0}: Error finding container 167cb7147e4f08312a51e2018ac039e7057dd77d730f6955248dea999912318f: Status 404 returned error can't find the container with id 167cb7147e4f08312a51e2018ac039e7057dd77d730f6955248dea999912318f Jul 28 17:14:06.317985 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:06.317955 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66jrd\" (UniqueName: \"kubernetes.io/projected/51f72926-f940-43a4-9187-4363dc1d7f50-kube-api-access-66jrd\") pod \"istiod-openshift-gateway-7b7995d5cd-n426d\" (UID: \"51f72926-f940-43a4-9187-4363dc1d7f50\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d" Jul 28 17:14:06.317985 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:06.317988 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/51f72926-f940-43a4-9187-4363dc1d7f50-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-n426d\" (UID: \"51f72926-f940-43a4-9187-4363dc1d7f50\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d" Jul 28 17:14:06.318220 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:06.318017 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/51f72926-f940-43a4-9187-4363dc1d7f50-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-n426d\" (UID: \"51f72926-f940-43a4-9187-4363dc1d7f50\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d" Jul 28 17:14:06.318220 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:06.318097 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/51f72926-f940-43a4-9187-4363dc1d7f50-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-n426d\" (UID: \"51f72926-f940-43a4-9187-4363dc1d7f50\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d" Jul 28 17:14:06.318220 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:06.318174 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/51f72926-f940-43a4-9187-4363dc1d7f50-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-n426d\" (UID: \"51f72926-f940-43a4-9187-4363dc1d7f50\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d" Jul 28 17:14:06.318220 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:06.318199 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51f72926-f940-43a4-9187-4363dc1d7f50-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-n426d\" (UID: \"51f72926-f940-43a4-9187-4363dc1d7f50\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d" Jul 28 17:14:06.318430 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:06.318229 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/51f72926-f940-43a4-9187-4363dc1d7f50-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-n426d\" (UID: \"51f72926-f940-43a4-9187-4363dc1d7f50\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d" Jul 28 17:14:06.419360 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:06.419327 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66jrd\" (UniqueName: \"kubernetes.io/projected/51f72926-f940-43a4-9187-4363dc1d7f50-kube-api-access-66jrd\") pod \"istiod-openshift-gateway-7b7995d5cd-n426d\" (UID: \"51f72926-f940-43a4-9187-4363dc1d7f50\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d" Jul 28 17:14:06.419506 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:06.419372 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/51f72926-f940-43a4-9187-4363dc1d7f50-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-n426d\" (UID: \"51f72926-f940-43a4-9187-4363dc1d7f50\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d" Jul 28 17:14:06.419506 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:06.419401 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/51f72926-f940-43a4-9187-4363dc1d7f50-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-n426d\" (UID: \"51f72926-f940-43a4-9187-4363dc1d7f50\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d" Jul 28 17:14:06.419612 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:06.419562 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/51f72926-f940-43a4-9187-4363dc1d7f50-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-n426d\" (UID: \"51f72926-f940-43a4-9187-4363dc1d7f50\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d" Jul 28 17:14:06.419664 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:06.419633 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/51f72926-f940-43a4-9187-4363dc1d7f50-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-n426d\" (UID: \"51f72926-f940-43a4-9187-4363dc1d7f50\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d" Jul 28 17:14:06.419712 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:06.419671 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51f72926-f940-43a4-9187-4363dc1d7f50-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-n426d\" (UID: \"51f72926-f940-43a4-9187-4363dc1d7f50\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d" Jul 28 17:14:06.419760 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:06.419709 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/51f72926-f940-43a4-9187-4363dc1d7f50-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-n426d\" (UID: \"51f72926-f940-43a4-9187-4363dc1d7f50\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d" Jul 28 17:14:06.420105 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:06.420061 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/51f72926-f940-43a4-9187-4363dc1d7f50-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-n426d\" (UID: \"51f72926-f940-43a4-9187-4363dc1d7f50\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d" Jul 28 17:14:06.422030 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:06.421987 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/51f72926-f940-43a4-9187-4363dc1d7f50-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-n426d\" (UID: \"51f72926-f940-43a4-9187-4363dc1d7f50\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d" Jul 28 17:14:06.422721 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:06.422675 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/51f72926-f940-43a4-9187-4363dc1d7f50-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-n426d\" (UID: \"51f72926-f940-43a4-9187-4363dc1d7f50\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d" Jul 28 17:14:06.422822 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:06.422752 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/51f72926-f940-43a4-9187-4363dc1d7f50-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-n426d\" (UID: \"51f72926-f940-43a4-9187-4363dc1d7f50\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d" Jul 28 17:14:06.422822 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:06.422807 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/51f72926-f940-43a4-9187-4363dc1d7f50-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-n426d\" (UID: \"51f72926-f940-43a4-9187-4363dc1d7f50\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d" Jul 28 17:14:06.437042 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:06.436984 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66jrd\" (UniqueName: \"kubernetes.io/projected/51f72926-f940-43a4-9187-4363dc1d7f50-kube-api-access-66jrd\") pod \"istiod-openshift-gateway-7b7995d5cd-n426d\" (UID: \"51f72926-f940-43a4-9187-4363dc1d7f50\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d" Jul 28 17:14:06.437810 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:06.437787 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51f72926-f940-43a4-9187-4363dc1d7f50-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-n426d\" (UID: \"51f72926-f940-43a4-9187-4363dc1d7f50\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d" Jul 28 17:14:06.529520 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:06.529483 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d" Jul 28 17:14:06.727845 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:06.727815 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d"] Jul 28 17:14:06.731732 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:14:06.731700 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51f72926_f940_43a4_9187_4363dc1d7f50.slice/crio-03de893fe1b4b6c25c01e33d7b5f05c3079642b689d01f76af45e5b50608d3ba WatchSource:0}: Error finding container 03de893fe1b4b6c25c01e33d7b5f05c3079642b689d01f76af45e5b50608d3ba: Status 404 returned error can't find the container with id 03de893fe1b4b6c25c01e33d7b5f05c3079642b689d01f76af45e5b50608d3ba Jul 28 17:14:06.782881 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:06.782847 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-7t86x" event={"ID":"9eb4c97f-be9f-4584-b563-9a7e7efc1d2a","Type":"ContainerStarted","Data":"167cb7147e4f08312a51e2018ac039e7057dd77d730f6955248dea999912318f"} Jul 28 17:14:06.784327 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:06.784295 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d" event={"ID":"51f72926-f940-43a4-9187-4363dc1d7f50","Type":"ContainerStarted","Data":"03de893fe1b4b6c25c01e33d7b5f05c3079642b689d01f76af45e5b50608d3ba"} Jul 28 17:14:10.599599 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:10.599564 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 17:14:10.599969 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:10.599645 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 17:14:10.807698 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:10.807611 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-77b678cfbc-gg5gz" event={"ID":"1d224fae-cf17-4bdb-baf5-1dde1cd43871","Type":"ContainerStarted","Data":"87ad8733c90105bc955a80aa1c2f2834adce75971490e16b9ab47c9a186ef9b0"} Jul 28 17:14:10.809290 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:10.809257 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-mpxpz" event={"ID":"650f3a82-4de5-4bd2-ba3b-4f9c4c666f53","Type":"ContainerStarted","Data":"fa19da4a7cd3a2cc2aae7d0af70ebc5dfa0b7abde6a165fe84999d55f78256e6"} Jul 28 17:14:10.809419 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:10.809332 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-mpxpz" Jul 28 17:14:10.810773 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:10.810747 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d" event={"ID":"51f72926-f940-43a4-9187-4363dc1d7f50","Type":"ContainerStarted","Data":"ffe39f08666a7cf629d58d4d97d008ea26452274fa00571b5f39e5afb11ca913"} Jul 28 17:14:10.810888 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:10.810797 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d" Jul 28 17:14:10.812254 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:10.812233 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-7t86x" event={"ID":"9eb4c97f-be9f-4584-b563-9a7e7efc1d2a","Type":"ContainerStarted","Data":"1ccdf8e9f0353c2a1e6c5c23a0538a95310edb758453ef644d35aa7ef721975f"} Jul 28 17:14:10.812369 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:10.812356 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-7t86x" Jul 28 17:14:10.851375 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:10.851325 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-77b678cfbc-gg5gz" podStartSLOduration=1.255353967 podStartE2EDuration="6.851311648s" totalImagesPullingTime="5.595957681s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:14:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:14:10.849918307 +0000 UTC m=+349.458849692" watchObservedRunningTime="2026-07-28 17:14:10.851311648 +0000 UTC m=+349.460243019" Jul 28 17:14:10.888759 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:10.888708 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d" podStartSLOduration=1.023787229 podStartE2EDuration="4.888669328s" totalImagesPullingTime="3.864882099s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:14:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:14:10.887235993 +0000 UTC m=+349.496167373" watchObservedRunningTime="2026-07-28 17:14:10.888669328 +0000 UTC m=+349.497600679" Jul 28 17:14:10.919597 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:10.919524 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-mpxpz" podStartSLOduration=2.099842413 podStartE2EDuration="6.919510386s" totalImagesPullingTime="4.819667973s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:14:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:14:10.918754983 +0000 UTC m=+349.527686352" watchObservedRunningTime="2026-07-28 17:14:10.919510386 +0000 UTC m=+349.528441756" Jul 28 17:14:11.818469 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:11.818437 2570 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-n426d container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 28 17:14:11.818874 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:11.818486 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d" podUID="51f72926-f940-43a4-9187-4363dc1d7f50" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:14:12.879130 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:12.879059 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-7t86x" podStartSLOduration=3.637727615 podStartE2EDuration="7.879041538s" totalImagesPullingTime="4.241313923s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:14:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:14:10.971037953 +0000 UTC m=+349.579969323" watchObservedRunningTime="2026-07-28 17:14:12.879041538 +0000 UTC m=+351.487972908" Jul 28 17:14:12.879783 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:12.879763 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7"] Jul 28 17:14:12.883273 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:12.883257 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7" Jul 28 17:14:12.886944 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:12.886922 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-2jrmf\"" Jul 28 17:14:12.919559 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:12.919532 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7"] Jul 28 17:14:12.980854 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:12.980822 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bbda9ba-84f0-40a5-979d-9ea282384e4e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7\" (UID: \"9bbda9ba-84f0-40a5-979d-9ea282384e4e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7" Jul 28 17:14:12.981011 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:12.980864 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bbda9ba-84f0-40a5-979d-9ea282384e4e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7\" (UID: \"9bbda9ba-84f0-40a5-979d-9ea282384e4e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7" Jul 28 17:14:12.981011 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:12.980892 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bbda9ba-84f0-40a5-979d-9ea282384e4e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7\" (UID: \"9bbda9ba-84f0-40a5-979d-9ea282384e4e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7" Jul 28 17:14:12.981011 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:12.980970 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bbda9ba-84f0-40a5-979d-9ea282384e4e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7\" (UID: \"9bbda9ba-84f0-40a5-979d-9ea282384e4e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7" Jul 28 17:14:12.981011 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:12.980993 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfs4g\" (UniqueName: \"kubernetes.io/projected/9bbda9ba-84f0-40a5-979d-9ea282384e4e-kube-api-access-sfs4g\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7\" (UID: \"9bbda9ba-84f0-40a5-979d-9ea282384e4e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7" Jul 28 17:14:12.981164 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:12.981036 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bbda9ba-84f0-40a5-979d-9ea282384e4e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7\" (UID: \"9bbda9ba-84f0-40a5-979d-9ea282384e4e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7" Jul 28 17:14:12.981164 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:12.981070 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bbda9ba-84f0-40a5-979d-9ea282384e4e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7\" (UID: \"9bbda9ba-84f0-40a5-979d-9ea282384e4e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7" Jul 28 17:14:12.981164 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:12.981093 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bbda9ba-84f0-40a5-979d-9ea282384e4e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7\" (UID: \"9bbda9ba-84f0-40a5-979d-9ea282384e4e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7" Jul 28 17:14:12.981164 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:12.981134 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bbda9ba-84f0-40a5-979d-9ea282384e4e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7\" (UID: \"9bbda9ba-84f0-40a5-979d-9ea282384e4e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7" Jul 28 17:14:13.086866 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:13.086829 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bbda9ba-84f0-40a5-979d-9ea282384e4e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7\" (UID: \"9bbda9ba-84f0-40a5-979d-9ea282384e4e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7" Jul 28 17:14:13.087036 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:13.086879 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bbda9ba-84f0-40a5-979d-9ea282384e4e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7\" (UID: \"9bbda9ba-84f0-40a5-979d-9ea282384e4e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7" Jul 28 17:14:13.087036 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:13.086901 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bbda9ba-84f0-40a5-979d-9ea282384e4e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7\" (UID: \"9bbda9ba-84f0-40a5-979d-9ea282384e4e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7" Jul 28 17:14:13.087036 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:13.086921 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bbda9ba-84f0-40a5-979d-9ea282384e4e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7\" (UID: \"9bbda9ba-84f0-40a5-979d-9ea282384e4e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7" Jul 28 17:14:13.087036 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:13.086958 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfs4g\" (UniqueName: \"kubernetes.io/projected/9bbda9ba-84f0-40a5-979d-9ea282384e4e-kube-api-access-sfs4g\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7\" (UID: \"9bbda9ba-84f0-40a5-979d-9ea282384e4e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7" Jul 28 17:14:13.087295 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:13.087042 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bbda9ba-84f0-40a5-979d-9ea282384e4e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7\" (UID: \"9bbda9ba-84f0-40a5-979d-9ea282384e4e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7" Jul 28 17:14:13.087295 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:13.087090 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bbda9ba-84f0-40a5-979d-9ea282384e4e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7\" (UID: \"9bbda9ba-84f0-40a5-979d-9ea282384e4e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7" Jul 28 17:14:13.087295 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:13.087141 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bbda9ba-84f0-40a5-979d-9ea282384e4e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7\" (UID: \"9bbda9ba-84f0-40a5-979d-9ea282384e4e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7" Jul 28 17:14:13.087295 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:13.087179 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bbda9ba-84f0-40a5-979d-9ea282384e4e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7\" (UID: \"9bbda9ba-84f0-40a5-979d-9ea282384e4e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7" Jul 28 17:14:13.087447 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:13.087295 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bbda9ba-84f0-40a5-979d-9ea282384e4e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7\" (UID: \"9bbda9ba-84f0-40a5-979d-9ea282384e4e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7" Jul 28 17:14:13.087447 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:13.087402 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bbda9ba-84f0-40a5-979d-9ea282384e4e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7\" (UID: \"9bbda9ba-84f0-40a5-979d-9ea282384e4e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7" Jul 28 17:14:13.087447 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:13.087430 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bbda9ba-84f0-40a5-979d-9ea282384e4e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7\" (UID: \"9bbda9ba-84f0-40a5-979d-9ea282384e4e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7" Jul 28 17:14:13.087658 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:13.087631 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bbda9ba-84f0-40a5-979d-9ea282384e4e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7\" (UID: \"9bbda9ba-84f0-40a5-979d-9ea282384e4e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7" Jul 28 17:14:13.087814 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:13.087792 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bbda9ba-84f0-40a5-979d-9ea282384e4e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7\" (UID: \"9bbda9ba-84f0-40a5-979d-9ea282384e4e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7" Jul 28 17:14:13.089393 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:13.089371 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bbda9ba-84f0-40a5-979d-9ea282384e4e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7\" (UID: \"9bbda9ba-84f0-40a5-979d-9ea282384e4e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7" Jul 28 17:14:13.089658 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:13.089641 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bbda9ba-84f0-40a5-979d-9ea282384e4e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7\" (UID: \"9bbda9ba-84f0-40a5-979d-9ea282384e4e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7" Jul 28 17:14:13.119223 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:13.119195 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfs4g\" (UniqueName: \"kubernetes.io/projected/9bbda9ba-84f0-40a5-979d-9ea282384e4e-kube-api-access-sfs4g\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7\" (UID: \"9bbda9ba-84f0-40a5-979d-9ea282384e4e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7" Jul 28 17:14:13.135238 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:13.135179 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bbda9ba-84f0-40a5-979d-9ea282384e4e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7\" (UID: \"9bbda9ba-84f0-40a5-979d-9ea282384e4e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7" Jul 28 17:14:13.194078 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:13.194047 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7" Jul 28 17:14:13.366640 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:13.366615 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7"] Jul 28 17:14:13.367282 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:14:13.367247 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bbda9ba_84f0_40a5_979d_9ea282384e4e.slice/crio-f072cae625ada77315d4678864282f7f4550a5e37b10ce003a900f7419f786fe WatchSource:0}: Error finding container f072cae625ada77315d4678864282f7f4550a5e37b10ce003a900f7419f786fe: Status 404 returned error can't find the container with id f072cae625ada77315d4678864282f7f4550a5e37b10ce003a900f7419f786fe Jul 28 17:14:13.825952 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:13.825904 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7" event={"ID":"9bbda9ba-84f0-40a5-979d-9ea282384e4e","Type":"ContainerStarted","Data":"f072cae625ada77315d4678864282f7f4550a5e37b10ce003a900f7419f786fe"} Jul 28 17:14:14.818182 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:14.818087 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d" Jul 28 17:14:16.713235 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:16.713203 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 17:14:16.713538 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:16.713271 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 17:14:16.713538 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:16.713301 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 17:14:16.843541 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:16.843501 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7" event={"ID":"9bbda9ba-84f0-40a5-979d-9ea282384e4e","Type":"ContainerStarted","Data":"f8af60f97466061c0737933a7a5f5a84c2d6d18c073583ec160f96f38e41c4e8"} Jul 28 17:14:16.972752 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:16.972656 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7" podStartSLOduration=1.6290352179999998 podStartE2EDuration="4.97264026s" totalImagesPullingTime="3.343605042s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:14:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:14:16.971287902 +0000 UTC m=+355.580219310" watchObservedRunningTime="2026-07-28 17:14:16.97264026 +0000 UTC m=+355.581571695" Jul 28 17:14:17.194549 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:17.194518 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7" Jul 28 17:14:17.199035 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:17.199011 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7" Jul 28 17:14:17.848731 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:17.848695 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7" Jul 28 17:14:17.849856 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:17.849838 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7" Jul 28 17:14:18.386964 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:18.386930 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-lmn47"] Jul 28 17:14:18.390385 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:18.390367 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lmn47" Jul 28 17:14:18.396744 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:18.396721 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 28 17:14:18.405081 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:18.405060 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-s9lmn"] Jul 28 17:14:18.408240 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:18.408223 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-s9lmn" Jul 28 17:14:18.411796 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:18.411777 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-lmn47"] Jul 28 17:14:18.413031 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:18.413013 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-k4mtv\"" Jul 28 17:14:18.413296 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:18.413278 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-wsq89\"" Jul 28 17:14:18.414844 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:18.414822 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 28 17:14:18.436749 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:18.436723 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pfpv\" (UniqueName: \"kubernetes.io/projected/039b79ed-2f6b-41ba-8529-fe1a6be83441-kube-api-access-6pfpv\") pod \"llmisvc-controller-manager-777c6bd7d5-lmn47\" (UID: \"039b79ed-2f6b-41ba-8529-fe1a6be83441\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lmn47" Jul 28 17:14:18.436870 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:18.436757 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/039b79ed-2f6b-41ba-8529-fe1a6be83441-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-lmn47\" (UID: \"039b79ed-2f6b-41ba-8529-fe1a6be83441\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lmn47" Jul 28 17:14:18.466780 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:18.466762 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-s9lmn"] Jul 28 17:14:18.538182 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:18.538143 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/039b79ed-2f6b-41ba-8529-fe1a6be83441-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-lmn47\" (UID: \"039b79ed-2f6b-41ba-8529-fe1a6be83441\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lmn47" Jul 28 17:14:18.538357 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:18.538226 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tlb8\" (UniqueName: \"kubernetes.io/projected/e988c6ed-5788-4f9c-91d3-263b35ab7d49-kube-api-access-8tlb8\") pod \"kserve-controller-manager-d6f4b65b8-s9lmn\" (UID: \"e988c6ed-5788-4f9c-91d3-263b35ab7d49\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-s9lmn" Jul 28 17:14:18.538357 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:18.538260 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e988c6ed-5788-4f9c-91d3-263b35ab7d49-cert\") pod \"kserve-controller-manager-d6f4b65b8-s9lmn\" (UID: \"e988c6ed-5788-4f9c-91d3-263b35ab7d49\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-s9lmn" Jul 28 17:14:18.538357 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:14:18.538264 2570 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 28 17:14:18.538357 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:18.538290 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pfpv\" (UniqueName: \"kubernetes.io/projected/039b79ed-2f6b-41ba-8529-fe1a6be83441-kube-api-access-6pfpv\") pod \"llmisvc-controller-manager-777c6bd7d5-lmn47\" (UID: \"039b79ed-2f6b-41ba-8529-fe1a6be83441\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lmn47" Jul 28 17:14:18.538357 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:14:18.538329 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/039b79ed-2f6b-41ba-8529-fe1a6be83441-cert podName:039b79ed-2f6b-41ba-8529-fe1a6be83441 nodeName:}" failed. No retries permitted until 2026-07-28 17:14:19.038311747 +0000 UTC m=+357.647243119 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/039b79ed-2f6b-41ba-8529-fe1a6be83441-cert") pod "llmisvc-controller-manager-777c6bd7d5-lmn47" (UID: "039b79ed-2f6b-41ba-8529-fe1a6be83441") : secret "llmisvc-webhook-server-cert" not found Jul 28 17:14:18.556392 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:18.556361 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pfpv\" (UniqueName: \"kubernetes.io/projected/039b79ed-2f6b-41ba-8529-fe1a6be83441-kube-api-access-6pfpv\") pod \"llmisvc-controller-manager-777c6bd7d5-lmn47\" (UID: \"039b79ed-2f6b-41ba-8529-fe1a6be83441\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lmn47" Jul 28 17:14:18.639669 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:18.639578 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tlb8\" (UniqueName: \"kubernetes.io/projected/e988c6ed-5788-4f9c-91d3-263b35ab7d49-kube-api-access-8tlb8\") pod \"kserve-controller-manager-d6f4b65b8-s9lmn\" (UID: \"e988c6ed-5788-4f9c-91d3-263b35ab7d49\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-s9lmn" Jul 28 17:14:18.639669 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:18.639636 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e988c6ed-5788-4f9c-91d3-263b35ab7d49-cert\") pod \"kserve-controller-manager-d6f4b65b8-s9lmn\" (UID: \"e988c6ed-5788-4f9c-91d3-263b35ab7d49\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-s9lmn" Jul 28 17:14:18.642193 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:18.642168 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e988c6ed-5788-4f9c-91d3-263b35ab7d49-cert\") pod \"kserve-controller-manager-d6f4b65b8-s9lmn\" (UID: \"e988c6ed-5788-4f9c-91d3-263b35ab7d49\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-s9lmn" Jul 28 17:14:18.651463 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:18.651438 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tlb8\" (UniqueName: \"kubernetes.io/projected/e988c6ed-5788-4f9c-91d3-263b35ab7d49-kube-api-access-8tlb8\") pod \"kserve-controller-manager-d6f4b65b8-s9lmn\" (UID: \"e988c6ed-5788-4f9c-91d3-263b35ab7d49\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-s9lmn" Jul 28 17:14:18.718484 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:18.718451 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-s9lmn" Jul 28 17:14:18.866148 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:18.865992 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-s9lmn"] Jul 28 17:14:18.868386 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:14:18.868361 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode988c6ed_5788_4f9c_91d3_263b35ab7d49.slice/crio-c51b8bc085c3890a923cc451b74ec2412748a79a18a1b71e4962ebcd5d5238da WatchSource:0}: Error finding container c51b8bc085c3890a923cc451b74ec2412748a79a18a1b71e4962ebcd5d5238da: Status 404 returned error can't find the container with id c51b8bc085c3890a923cc451b74ec2412748a79a18a1b71e4962ebcd5d5238da Jul 28 17:14:19.043226 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:19.043141 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/039b79ed-2f6b-41ba-8529-fe1a6be83441-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-lmn47\" (UID: \"039b79ed-2f6b-41ba-8529-fe1a6be83441\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lmn47" Jul 28 17:14:19.043371 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:14:19.043279 2570 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 28 17:14:19.043371 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:14:19.043339 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/039b79ed-2f6b-41ba-8529-fe1a6be83441-cert podName:039b79ed-2f6b-41ba-8529-fe1a6be83441 nodeName:}" failed. No retries permitted until 2026-07-28 17:14:20.04332457 +0000 UTC m=+358.652255921 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/039b79ed-2f6b-41ba-8529-fe1a6be83441-cert") pod "llmisvc-controller-manager-777c6bd7d5-lmn47" (UID: "039b79ed-2f6b-41ba-8529-fe1a6be83441") : secret "llmisvc-webhook-server-cert" not found Jul 28 17:14:19.859761 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:19.859726 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-s9lmn" event={"ID":"e988c6ed-5788-4f9c-91d3-263b35ab7d49","Type":"ContainerStarted","Data":"c51b8bc085c3890a923cc451b74ec2412748a79a18a1b71e4962ebcd5d5238da"} Jul 28 17:14:20.052714 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:20.052673 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/039b79ed-2f6b-41ba-8529-fe1a6be83441-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-lmn47\" (UID: \"039b79ed-2f6b-41ba-8529-fe1a6be83441\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lmn47" Jul 28 17:14:20.055067 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:20.055039 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/039b79ed-2f6b-41ba-8529-fe1a6be83441-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-lmn47\" (UID: \"039b79ed-2f6b-41ba-8529-fe1a6be83441\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lmn47" Jul 28 17:14:20.199975 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:20.199948 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lmn47" Jul 28 17:14:20.300641 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:20.300608 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d"] Jul 28 17:14:20.300858 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:20.300832 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d" podUID="51f72926-f940-43a4-9187-4363dc1d7f50" containerName="discovery" containerID="cri-o://ffe39f08666a7cf629d58d4d97d008ea26452274fa00571b5f39e5afb11ca913" gracePeriod=30 Jul 28 17:14:20.380398 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:20.380370 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-lmn47"] Jul 28 17:14:20.390049 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:14:20.390013 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod039b79ed_2f6b_41ba_8529_fe1a6be83441.slice/crio-f211933472f51e5bdc2ef1c76e1da75d9c0e39c1d01ef71a2f9b4ee634853a8d WatchSource:0}: Error finding container f211933472f51e5bdc2ef1c76e1da75d9c0e39c1d01ef71a2f9b4ee634853a8d: Status 404 returned error can't find the container with id f211933472f51e5bdc2ef1c76e1da75d9c0e39c1d01ef71a2f9b4ee634853a8d Jul 28 17:14:20.546771 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:20.546689 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d" Jul 28 17:14:20.659153 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:20.659094 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/51f72926-f940-43a4-9187-4363dc1d7f50-cacerts\") pod \"51f72926-f940-43a4-9187-4363dc1d7f50\" (UID: \"51f72926-f940-43a4-9187-4363dc1d7f50\") " Jul 28 17:14:20.659337 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:20.659172 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51f72926-f940-43a4-9187-4363dc1d7f50-istio-token\") pod \"51f72926-f940-43a4-9187-4363dc1d7f50\" (UID: \"51f72926-f940-43a4-9187-4363dc1d7f50\") " Jul 28 17:14:20.659337 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:20.659206 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/51f72926-f940-43a4-9187-4363dc1d7f50-istio-kubeconfig\") pod \"51f72926-f940-43a4-9187-4363dc1d7f50\" (UID: \"51f72926-f940-43a4-9187-4363dc1d7f50\") " Jul 28 17:14:20.659337 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:20.659273 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/51f72926-f940-43a4-9187-4363dc1d7f50-local-certs\") pod \"51f72926-f940-43a4-9187-4363dc1d7f50\" (UID: \"51f72926-f940-43a4-9187-4363dc1d7f50\") " Jul 28 17:14:20.659337 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:20.659314 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/51f72926-f940-43a4-9187-4363dc1d7f50-istio-csr-dns-cert\") pod \"51f72926-f940-43a4-9187-4363dc1d7f50\" (UID: \"51f72926-f940-43a4-9187-4363dc1d7f50\") " Jul 28 17:14:20.659560 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:20.659359 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66jrd\" (UniqueName: \"kubernetes.io/projected/51f72926-f940-43a4-9187-4363dc1d7f50-kube-api-access-66jrd\") pod \"51f72926-f940-43a4-9187-4363dc1d7f50\" (UID: \"51f72926-f940-43a4-9187-4363dc1d7f50\") " Jul 28 17:14:20.659560 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:20.659393 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/51f72926-f940-43a4-9187-4363dc1d7f50-istio-csr-ca-configmap\") pod \"51f72926-f940-43a4-9187-4363dc1d7f50\" (UID: \"51f72926-f940-43a4-9187-4363dc1d7f50\") " Jul 28 17:14:20.660274 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:20.660245 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51f72926-f940-43a4-9187-4363dc1d7f50-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "51f72926-f940-43a4-9187-4363dc1d7f50" (UID: "51f72926-f940-43a4-9187-4363dc1d7f50"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:14:20.662247 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:20.662216 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51f72926-f940-43a4-9187-4363dc1d7f50-local-certs" (OuterVolumeSpecName: "local-certs") pod "51f72926-f940-43a4-9187-4363dc1d7f50" (UID: "51f72926-f940-43a4-9187-4363dc1d7f50"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:14:20.662377 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:20.662348 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51f72926-f940-43a4-9187-4363dc1d7f50-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "51f72926-f940-43a4-9187-4363dc1d7f50" (UID: "51f72926-f940-43a4-9187-4363dc1d7f50"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:14:20.662620 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:20.662597 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51f72926-f940-43a4-9187-4363dc1d7f50-istio-token" (OuterVolumeSpecName: "istio-token") pod "51f72926-f940-43a4-9187-4363dc1d7f50" (UID: "51f72926-f940-43a4-9187-4363dc1d7f50"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:14:20.662756 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:20.662737 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51f72926-f940-43a4-9187-4363dc1d7f50-cacerts" (OuterVolumeSpecName: "cacerts") pod "51f72926-f940-43a4-9187-4363dc1d7f50" (UID: "51f72926-f940-43a4-9187-4363dc1d7f50"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:14:20.662923 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:20.662900 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51f72926-f940-43a4-9187-4363dc1d7f50-kube-api-access-66jrd" (OuterVolumeSpecName: "kube-api-access-66jrd") pod "51f72926-f940-43a4-9187-4363dc1d7f50" (UID: "51f72926-f940-43a4-9187-4363dc1d7f50"). InnerVolumeSpecName "kube-api-access-66jrd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:14:20.663595 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:20.663563 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51f72926-f940-43a4-9187-4363dc1d7f50-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "51f72926-f940-43a4-9187-4363dc1d7f50" (UID: "51f72926-f940-43a4-9187-4363dc1d7f50"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:14:20.760164 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:20.760067 2570 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/51f72926-f940-43a4-9187-4363dc1d7f50-cacerts\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:14:20.760164 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:20.760094 2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51f72926-f940-43a4-9187-4363dc1d7f50-istio-token\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:14:20.760164 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:20.760155 2570 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/51f72926-f940-43a4-9187-4363dc1d7f50-istio-kubeconfig\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:14:20.760374 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:20.760168 2570 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/51f72926-f940-43a4-9187-4363dc1d7f50-local-certs\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:14:20.760374 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:20.760180 2570 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/51f72926-f940-43a4-9187-4363dc1d7f50-istio-csr-dns-cert\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:14:20.760374 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:20.760191 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66jrd\" (UniqueName: \"kubernetes.io/projected/51f72926-f940-43a4-9187-4363dc1d7f50-kube-api-access-66jrd\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:14:20.760374 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:20.760204 2570 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/51f72926-f940-43a4-9187-4363dc1d7f50-istio-csr-ca-configmap\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:14:20.865786 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:20.865755 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lmn47" event={"ID":"039b79ed-2f6b-41ba-8529-fe1a6be83441","Type":"ContainerStarted","Data":"f211933472f51e5bdc2ef1c76e1da75d9c0e39c1d01ef71a2f9b4ee634853a8d"} Jul 28 17:14:20.866953 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:20.866931 2570 generic.go:356] "Generic (PLEG): container finished" podID="51f72926-f940-43a4-9187-4363dc1d7f50" containerID="ffe39f08666a7cf629d58d4d97d008ea26452274fa00571b5f39e5afb11ca913" exitCode=0 Jul 28 17:14:20.867057 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:20.866975 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d" Jul 28 17:14:20.867057 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:20.867012 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d" event={"ID":"51f72926-f940-43a4-9187-4363dc1d7f50","Type":"ContainerDied","Data":"ffe39f08666a7cf629d58d4d97d008ea26452274fa00571b5f39e5afb11ca913"} Jul 28 17:14:20.867152 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:20.867058 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d" event={"ID":"51f72926-f940-43a4-9187-4363dc1d7f50","Type":"ContainerDied","Data":"03de893fe1b4b6c25c01e33d7b5f05c3079642b689d01f76af45e5b50608d3ba"} Jul 28 17:14:20.867152 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:20.867085 2570 scope.go:117] "RemoveContainer" containerID="ffe39f08666a7cf629d58d4d97d008ea26452274fa00571b5f39e5afb11ca913" Jul 28 17:14:20.878701 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:20.878684 2570 scope.go:117] "RemoveContainer" containerID="ffe39f08666a7cf629d58d4d97d008ea26452274fa00571b5f39e5afb11ca913" Jul 28 17:14:20.879360 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:14:20.879308 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffe39f08666a7cf629d58d4d97d008ea26452274fa00571b5f39e5afb11ca913\": container with ID starting with ffe39f08666a7cf629d58d4d97d008ea26452274fa00571b5f39e5afb11ca913 not found: ID does not exist" containerID="ffe39f08666a7cf629d58d4d97d008ea26452274fa00571b5f39e5afb11ca913" Jul 28 17:14:20.879481 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:20.879368 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffe39f08666a7cf629d58d4d97d008ea26452274fa00571b5f39e5afb11ca913"} err="failed to get container status \"ffe39f08666a7cf629d58d4d97d008ea26452274fa00571b5f39e5afb11ca913\": rpc error: code = NotFound desc = could not find container \"ffe39f08666a7cf629d58d4d97d008ea26452274fa00571b5f39e5afb11ca913\": container with ID starting with ffe39f08666a7cf629d58d4d97d008ea26452274fa00571b5f39e5afb11ca913 not found: ID does not exist" Jul 28 17:14:20.902993 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:20.902970 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d"] Jul 28 17:14:20.916917 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:20.916895 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n426d"] Jul 28 17:14:21.819169 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:21.819136 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-mpxpz" Jul 28 17:14:21.822288 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:21.822261 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-7t86x" Jul 28 17:14:22.120469 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:22.120444 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51f72926-f940-43a4-9187-4363dc1d7f50" path="/var/lib/kubelet/pods/51f72926-f940-43a4-9187-4363dc1d7f50/volumes" Jul 28 17:14:22.878532 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:22.878499 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-s9lmn" event={"ID":"e988c6ed-5788-4f9c-91d3-263b35ab7d49","Type":"ContainerStarted","Data":"f77c5fbd62e7079048145d4ac1afc67aade9ed5c0e1a02ec4bfecd06632fe154"} Jul 28 17:14:22.879013 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:22.878658 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-s9lmn" Jul 28 17:14:24.888159 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:24.888104 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lmn47" event={"ID":"039b79ed-2f6b-41ba-8529-fe1a6be83441","Type":"ContainerStarted","Data":"23c5f4b89e2e659dae5ba8e60bf0106fc8d2b1782821aa95aaf1af99c662ef23"} Jul 28 17:14:24.888555 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:24.888209 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lmn47" Jul 28 17:14:24.918369 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:24.918320 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lmn47" podStartSLOduration=2.741934474 podStartE2EDuration="6.91830363s" totalImagesPullingTime="4.176369156s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:14:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:14:24.91598276 +0000 UTC m=+363.524914148" watchObservedRunningTime="2026-07-28 17:14:24.91830363 +0000 UTC m=+363.527235017" Jul 28 17:14:24.918534 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:24.918396 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-s9lmn" podStartSLOduration=3.571706229 podStartE2EDuration="6.918392122s" totalImagesPullingTime="3.346685893s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:14:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:14:22.91340098 +0000 UTC m=+361.522332369" watchObservedRunningTime="2026-07-28 17:14:24.918392122 +0000 UTC m=+363.527323492" Jul 28 17:14:53.888447 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:53.888413 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-s9lmn" Jul 28 17:14:55.895240 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:14:55.895209 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lmn47" Jul 28 17:15:05.051501 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:05.051467 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j645w"] Jul 28 17:15:05.051970 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:05.051935 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51f72926-f940-43a4-9187-4363dc1d7f50" containerName="discovery" Jul 28 17:15:05.051970 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:05.051953 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="51f72926-f940-43a4-9187-4363dc1d7f50" containerName="discovery" Jul 28 17:15:05.052143 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:05.052066 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="51f72926-f940-43a4-9187-4363dc1d7f50" containerName="discovery" Jul 28 17:15:05.058154 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:05.058099 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-j645w" Jul 28 17:15:05.061206 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:05.061173 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 28 17:15:05.061329 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:05.061211 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 28 17:15:05.062319 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:05.062296 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-6vxgj\"" Jul 28 17:15:05.067255 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:05.067232 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j645w"] Jul 28 17:15:05.140717 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:05.140684 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7c8f\" (UniqueName: \"kubernetes.io/projected/17ae812b-2051-42db-a163-19047353223d-kube-api-access-z7c8f\") pod \"kuadrant-operator-catalog-j645w\" (UID: \"17ae812b-2051-42db-a163-19047353223d\") " pod="kuadrant-system/kuadrant-operator-catalog-j645w" Jul 28 17:15:05.241811 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:05.241777 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7c8f\" (UniqueName: \"kubernetes.io/projected/17ae812b-2051-42db-a163-19047353223d-kube-api-access-z7c8f\") pod \"kuadrant-operator-catalog-j645w\" (UID: \"17ae812b-2051-42db-a163-19047353223d\") " pod="kuadrant-system/kuadrant-operator-catalog-j645w" Jul 28 17:15:05.251386 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:05.251354 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7c8f\" (UniqueName: \"kubernetes.io/projected/17ae812b-2051-42db-a163-19047353223d-kube-api-access-z7c8f\") pod \"kuadrant-operator-catalog-j645w\" (UID: \"17ae812b-2051-42db-a163-19047353223d\") " pod="kuadrant-system/kuadrant-operator-catalog-j645w" Jul 28 17:15:05.368985 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:05.368904 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-j645w" Jul 28 17:15:05.409617 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:05.409587 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j645w"] Jul 28 17:15:05.497487 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:05.497461 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j645w"] Jul 28 17:15:05.498800 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:15:05.498772 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17ae812b_2051_42db_a163_19047353223d.slice/crio-870a79f5445d87a4f6f6a77580a4534c02f378fadf631684c28113e29af6ba12 WatchSource:0}: Error finding container 870a79f5445d87a4f6f6a77580a4534c02f378fadf631684c28113e29af6ba12: Status 404 returned error can't find the container with id 870a79f5445d87a4f6f6a77580a4534c02f378fadf631684c28113e29af6ba12 Jul 28 17:15:05.625867 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:05.625832 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r85k2"] Jul 28 17:15:05.628439 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:05.628422 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-r85k2" Jul 28 17:15:05.640283 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:05.640262 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r85k2"] Jul 28 17:15:05.745177 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:05.745142 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4lgp\" (UniqueName: \"kubernetes.io/projected/33717d5a-d5aa-4b97-bcec-a56118a8afee-kube-api-access-b4lgp\") pod \"kuadrant-operator-catalog-r85k2\" (UID: \"33717d5a-d5aa-4b97-bcec-a56118a8afee\") " pod="kuadrant-system/kuadrant-operator-catalog-r85k2" Jul 28 17:15:05.846255 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:05.846224 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4lgp\" (UniqueName: \"kubernetes.io/projected/33717d5a-d5aa-4b97-bcec-a56118a8afee-kube-api-access-b4lgp\") pod \"kuadrant-operator-catalog-r85k2\" (UID: \"33717d5a-d5aa-4b97-bcec-a56118a8afee\") " pod="kuadrant-system/kuadrant-operator-catalog-r85k2" Jul 28 17:15:05.856428 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:05.856399 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4lgp\" (UniqueName: \"kubernetes.io/projected/33717d5a-d5aa-4b97-bcec-a56118a8afee-kube-api-access-b4lgp\") pod \"kuadrant-operator-catalog-r85k2\" (UID: \"33717d5a-d5aa-4b97-bcec-a56118a8afee\") " pod="kuadrant-system/kuadrant-operator-catalog-r85k2" Jul 28 17:15:05.937395 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:05.937292 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-r85k2" Jul 28 17:15:06.068760 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:06.068722 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-j645w" event={"ID":"17ae812b-2051-42db-a163-19047353223d","Type":"ContainerStarted","Data":"870a79f5445d87a4f6f6a77580a4534c02f378fadf631684c28113e29af6ba12"} Jul 28 17:15:06.073392 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:06.073369 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r85k2"] Jul 28 17:15:06.075367 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:15:06.075341 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33717d5a_d5aa_4b97_bcec_a56118a8afee.slice/crio-8dd21e4af050cb8b117691efeb36bc672a029e733005a782e5b8a12454de3e45 WatchSource:0}: Error finding container 8dd21e4af050cb8b117691efeb36bc672a029e733005a782e5b8a12454de3e45: Status 404 returned error can't find the container with id 8dd21e4af050cb8b117691efeb36bc672a029e733005a782e5b8a12454de3e45 Jul 28 17:15:07.078088 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:07.078056 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-r85k2" event={"ID":"33717d5a-d5aa-4b97-bcec-a56118a8afee","Type":"ContainerStarted","Data":"8dd21e4af050cb8b117691efeb36bc672a029e733005a782e5b8a12454de3e45"} Jul 28 17:15:08.084736 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:08.084644 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-r85k2" event={"ID":"33717d5a-d5aa-4b97-bcec-a56118a8afee","Type":"ContainerStarted","Data":"d5d783a842f4b11d2a2e94d9d255bed33cf9c776fa09c0f8b814fe86f02f2336"} Jul 28 17:15:08.086172 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:08.086149 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-j645w" event={"ID":"17ae812b-2051-42db-a163-19047353223d","Type":"ContainerStarted","Data":"88b97d26717fff4c1e078cf7e3f65187d624aa75a6014e0a8f15ffd82af74245"} Jul 28 17:15:08.086281 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:08.086232 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-j645w" podUID="17ae812b-2051-42db-a163-19047353223d" containerName="registry-server" containerID="cri-o://88b97d26717fff4c1e078cf7e3f65187d624aa75a6014e0a8f15ffd82af74245" gracePeriod=2 Jul 28 17:15:08.108388 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:08.108345 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-r85k2" podStartSLOduration=1.5498723970000001 podStartE2EDuration="3.108324283s" totalImagesPullingTime="1.558451886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:15:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:15:08.107587731 +0000 UTC m=+406.716519127" watchObservedRunningTime="2026-07-28 17:15:08.108324283 +0000 UTC m=+406.717255652" Jul 28 17:15:08.127314 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:08.127277 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-j645w" podStartSLOduration=0.995632439 podStartE2EDuration="3.127264383s" totalImagesPullingTime="2.131631944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:15:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:15:08.125829157 +0000 UTC m=+406.734760527" watchObservedRunningTime="2026-07-28 17:15:08.127264383 +0000 UTC m=+406.736195752" Jul 28 17:15:08.322284 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:08.322262 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-j645w" Jul 28 17:15:08.368046 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:08.367962 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7c8f\" (UniqueName: \"kubernetes.io/projected/17ae812b-2051-42db-a163-19047353223d-kube-api-access-z7c8f\") pod \"17ae812b-2051-42db-a163-19047353223d\" (UID: \"17ae812b-2051-42db-a163-19047353223d\") " Jul 28 17:15:08.370147 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:08.370088 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17ae812b-2051-42db-a163-19047353223d-kube-api-access-z7c8f" (OuterVolumeSpecName: "kube-api-access-z7c8f") pod "17ae812b-2051-42db-a163-19047353223d" (UID: "17ae812b-2051-42db-a163-19047353223d"). InnerVolumeSpecName "kube-api-access-z7c8f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:15:08.468721 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:08.468690 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7c8f\" (UniqueName: \"kubernetes.io/projected/17ae812b-2051-42db-a163-19047353223d-kube-api-access-z7c8f\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:15:09.092254 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:09.092160 2570 generic.go:356] "Generic (PLEG): container finished" podID="17ae812b-2051-42db-a163-19047353223d" containerID="88b97d26717fff4c1e078cf7e3f65187d624aa75a6014e0a8f15ffd82af74245" exitCode=0 Jul 28 17:15:09.092254 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:09.092208 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-j645w" Jul 28 17:15:09.092254 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:09.092241 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-j645w" event={"ID":"17ae812b-2051-42db-a163-19047353223d","Type":"ContainerDied","Data":"88b97d26717fff4c1e078cf7e3f65187d624aa75a6014e0a8f15ffd82af74245"} Jul 28 17:15:09.092792 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:09.092291 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-j645w" event={"ID":"17ae812b-2051-42db-a163-19047353223d","Type":"ContainerDied","Data":"870a79f5445d87a4f6f6a77580a4534c02f378fadf631684c28113e29af6ba12"} Jul 28 17:15:09.092792 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:09.092310 2570 scope.go:117] "RemoveContainer" containerID="88b97d26717fff4c1e078cf7e3f65187d624aa75a6014e0a8f15ffd82af74245" Jul 28 17:15:09.101483 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:09.101467 2570 scope.go:117] "RemoveContainer" containerID="88b97d26717fff4c1e078cf7e3f65187d624aa75a6014e0a8f15ffd82af74245" Jul 28 17:15:09.101738 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:15:09.101720 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88b97d26717fff4c1e078cf7e3f65187d624aa75a6014e0a8f15ffd82af74245\": container with ID starting with 88b97d26717fff4c1e078cf7e3f65187d624aa75a6014e0a8f15ffd82af74245 not found: ID does not exist" containerID="88b97d26717fff4c1e078cf7e3f65187d624aa75a6014e0a8f15ffd82af74245" Jul 28 17:15:09.101783 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:09.101748 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88b97d26717fff4c1e078cf7e3f65187d624aa75a6014e0a8f15ffd82af74245"} err="failed to get container status \"88b97d26717fff4c1e078cf7e3f65187d624aa75a6014e0a8f15ffd82af74245\": rpc error: code = NotFound desc = could not find container \"88b97d26717fff4c1e078cf7e3f65187d624aa75a6014e0a8f15ffd82af74245\": container with ID starting with 88b97d26717fff4c1e078cf7e3f65187d624aa75a6014e0a8f15ffd82af74245 not found: ID does not exist" Jul 28 17:15:09.116617 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:09.116594 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j645w"] Jul 28 17:15:09.123824 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:09.123802 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j645w"] Jul 28 17:15:10.118413 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:10.118383 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17ae812b-2051-42db-a163-19047353223d" path="/var/lib/kubelet/pods/17ae812b-2051-42db-a163-19047353223d/volumes" Jul 28 17:15:15.937480 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:15.937442 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-r85k2" Jul 28 17:15:15.937856 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:15.937511 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-r85k2" Jul 28 17:15:15.959364 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:15.959339 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-r85k2" Jul 28 17:15:16.145650 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:16.145625 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-r85k2" Jul 28 17:15:37.352913 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:37.352874 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-wvxrn"] Jul 28 17:15:37.353398 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:37.353255 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17ae812b-2051-42db-a163-19047353223d" containerName="registry-server" Jul 28 17:15:37.353398 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:37.353267 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="17ae812b-2051-42db-a163-19047353223d" containerName="registry-server" Jul 28 17:15:37.353398 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:37.353321 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="17ae812b-2051-42db-a163-19047353223d" containerName="registry-server" Jul 28 17:15:37.357466 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:37.357447 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wvxrn" Jul 28 17:15:37.362593 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:37.362560 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-8jg8r\"" Jul 28 17:15:37.374985 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:37.374959 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-wvxrn"] Jul 28 17:15:37.416017 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:37.415976 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ws8r5\" (UniqueName: \"kubernetes.io/projected/b5723932-0f49-4854-9a46-8ab1f3fdec8d-kube-api-access-ws8r5\") pod \"authorino-operator-75cc8d5cb7-wvxrn\" (UID: \"b5723932-0f49-4854-9a46-8ab1f3fdec8d\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-wvxrn" Jul 28 17:15:37.517260 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:37.517214 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ws8r5\" (UniqueName: \"kubernetes.io/projected/b5723932-0f49-4854-9a46-8ab1f3fdec8d-kube-api-access-ws8r5\") pod \"authorino-operator-75cc8d5cb7-wvxrn\" (UID: \"b5723932-0f49-4854-9a46-8ab1f3fdec8d\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-wvxrn" Jul 28 17:15:37.529860 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:37.529828 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ws8r5\" (UniqueName: \"kubernetes.io/projected/b5723932-0f49-4854-9a46-8ab1f3fdec8d-kube-api-access-ws8r5\") pod \"authorino-operator-75cc8d5cb7-wvxrn\" (UID: \"b5723932-0f49-4854-9a46-8ab1f3fdec8d\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-wvxrn" Jul 28 17:15:37.667989 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:37.667956 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wvxrn" Jul 28 17:15:37.813577 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:37.810671 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-wvxrn"] Jul 28 17:15:38.215731 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:38.215696 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wvxrn" event={"ID":"b5723932-0f49-4854-9a46-8ab1f3fdec8d","Type":"ContainerStarted","Data":"184278c248fd1dd4aa0d61f56c887c4af83524c3155e470a4eb853111b516a70"} Jul 28 17:15:40.227208 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:40.227174 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wvxrn" event={"ID":"b5723932-0f49-4854-9a46-8ab1f3fdec8d","Type":"ContainerStarted","Data":"80dd250307b88d4ef192d98681b9d9bd4fff8e2fd8858aad1cdd6a567a95f43f"} Jul 28 17:15:40.227588 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:40.227333 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wvxrn" Jul 28 17:15:40.280190 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:40.280081 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wvxrn" podStartSLOduration=1.108936089 podStartE2EDuration="3.280066748s" totalImagesPullingTime="2.171130659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:15:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:15:40.276161589 +0000 UTC m=+438.885092959" watchObservedRunningTime="2026-07-28 17:15:40.280066748 +0000 UTC m=+438.888998117" Jul 28 17:15:41.968459 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:41.968421 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sn2x6"] Jul 28 17:15:41.975803 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:41.975773 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sn2x6" Jul 28 17:15:41.978588 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:41.978567 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-pb6mp\"" Jul 28 17:15:42.009143 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:42.006499 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sn2x6"] Jul 28 17:15:42.055620 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:42.055587 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/03a90820-6e21-44c0-be21-b7f42bbd6837-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sn2x6\" (UID: \"03a90820-6e21-44c0-be21-b7f42bbd6837\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sn2x6" Jul 28 17:15:42.055620 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:42.055625 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8tjd\" (UniqueName: \"kubernetes.io/projected/03a90820-6e21-44c0-be21-b7f42bbd6837-kube-api-access-k8tjd\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sn2x6\" (UID: \"03a90820-6e21-44c0-be21-b7f42bbd6837\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sn2x6" Jul 28 17:15:42.156970 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:42.156931 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/03a90820-6e21-44c0-be21-b7f42bbd6837-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sn2x6\" (UID: \"03a90820-6e21-44c0-be21-b7f42bbd6837\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sn2x6" Jul 28 17:15:42.157213 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:42.156983 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8tjd\" (UniqueName: \"kubernetes.io/projected/03a90820-6e21-44c0-be21-b7f42bbd6837-kube-api-access-k8tjd\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sn2x6\" (UID: \"03a90820-6e21-44c0-be21-b7f42bbd6837\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sn2x6" Jul 28 17:15:42.157405 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:42.157384 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/03a90820-6e21-44c0-be21-b7f42bbd6837-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sn2x6\" (UID: \"03a90820-6e21-44c0-be21-b7f42bbd6837\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sn2x6" Jul 28 17:15:42.169017 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:42.168985 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8tjd\" (UniqueName: \"kubernetes.io/projected/03a90820-6e21-44c0-be21-b7f42bbd6837-kube-api-access-k8tjd\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sn2x6\" (UID: \"03a90820-6e21-44c0-be21-b7f42bbd6837\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sn2x6" Jul 28 17:15:42.288145 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:42.288037 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sn2x6" Jul 28 17:15:42.431773 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:42.431748 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sn2x6"] Jul 28 17:15:42.434444 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:15:42.434407 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03a90820_6e21_44c0_be21_b7f42bbd6837.slice/crio-0d5aa5b237659965ed6d9c279f42e4ad41b292db220eee50b297b6e1782045a3 WatchSource:0}: Error finding container 0d5aa5b237659965ed6d9c279f42e4ad41b292db220eee50b297b6e1782045a3: Status 404 returned error can't find the container with id 0d5aa5b237659965ed6d9c279f42e4ad41b292db220eee50b297b6e1782045a3 Jul 28 17:15:43.241075 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:43.241038 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sn2x6" event={"ID":"03a90820-6e21-44c0-be21-b7f42bbd6837","Type":"ContainerStarted","Data":"0d5aa5b237659965ed6d9c279f42e4ad41b292db220eee50b297b6e1782045a3"} Jul 28 17:15:43.954153 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:43.954104 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-4z5f5"] Jul 28 17:15:43.957391 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:43.957375 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4z5f5" Jul 28 17:15:43.960294 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:43.960268 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-xc2fj\"" Jul 28 17:15:43.960796 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:43.960779 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 28 17:15:43.969989 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:43.969967 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfjp7\" (UniqueName: \"kubernetes.io/projected/776d3861-f435-4f91-b0d6-b4d3a1124820-kube-api-access-cfjp7\") pod \"dns-operator-controller-manager-864dd78c4c-4z5f5\" (UID: \"776d3861-f435-4f91-b0d6-b4d3a1124820\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4z5f5" Jul 28 17:15:43.972069 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:43.972036 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-4z5f5"] Jul 28 17:15:44.071219 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:44.071185 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfjp7\" (UniqueName: \"kubernetes.io/projected/776d3861-f435-4f91-b0d6-b4d3a1124820-kube-api-access-cfjp7\") pod \"dns-operator-controller-manager-864dd78c4c-4z5f5\" (UID: \"776d3861-f435-4f91-b0d6-b4d3a1124820\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4z5f5" Jul 28 17:15:44.085862 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:44.085836 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfjp7\" (UniqueName: \"kubernetes.io/projected/776d3861-f435-4f91-b0d6-b4d3a1124820-kube-api-access-cfjp7\") pod \"dns-operator-controller-manager-864dd78c4c-4z5f5\" (UID: \"776d3861-f435-4f91-b0d6-b4d3a1124820\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4z5f5" Jul 28 17:15:44.268052 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:44.267943 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4z5f5" Jul 28 17:15:44.422975 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:44.422946 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-4z5f5"] Jul 28 17:15:44.425105 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:15:44.425074 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod776d3861_f435_4f91_b0d6_b4d3a1124820.slice/crio-09a71f0e74dae1dd76d8df2e911d4aadf035f2f05ef8e0724f4063bda41f2e8d WatchSource:0}: Error finding container 09a71f0e74dae1dd76d8df2e911d4aadf035f2f05ef8e0724f4063bda41f2e8d: Status 404 returned error can't find the container with id 09a71f0e74dae1dd76d8df2e911d4aadf035f2f05ef8e0724f4063bda41f2e8d Jul 28 17:15:45.256092 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:45.256021 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4z5f5" event={"ID":"776d3861-f435-4f91-b0d6-b4d3a1124820","Type":"ContainerStarted","Data":"09a71f0e74dae1dd76d8df2e911d4aadf035f2f05ef8e0724f4063bda41f2e8d"} Jul 28 17:15:46.771010 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:46.770979 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v72wr"] Jul 28 17:15:46.774922 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:46.774900 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v72wr" Jul 28 17:15:46.777915 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:46.777889 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-h58ll\"" Jul 28 17:15:46.789146 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:46.789095 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v72wr"] Jul 28 17:15:46.796632 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:46.796563 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9tcd\" (UniqueName: \"kubernetes.io/projected/ad0332c7-5823-4e9a-9b43-ab4cacbc8449-kube-api-access-r9tcd\") pod \"limitador-operator-controller-manager-859d7b78bd-v72wr\" (UID: \"ad0332c7-5823-4e9a-9b43-ab4cacbc8449\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v72wr" Jul 28 17:15:46.897629 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:46.897591 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9tcd\" (UniqueName: \"kubernetes.io/projected/ad0332c7-5823-4e9a-9b43-ab4cacbc8449-kube-api-access-r9tcd\") pod \"limitador-operator-controller-manager-859d7b78bd-v72wr\" (UID: \"ad0332c7-5823-4e9a-9b43-ab4cacbc8449\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v72wr" Jul 28 17:15:46.913034 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:46.913004 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9tcd\" (UniqueName: \"kubernetes.io/projected/ad0332c7-5823-4e9a-9b43-ab4cacbc8449-kube-api-access-r9tcd\") pod \"limitador-operator-controller-manager-859d7b78bd-v72wr\" (UID: \"ad0332c7-5823-4e9a-9b43-ab4cacbc8449\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v72wr" Jul 28 17:15:47.087586 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:47.087509 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v72wr" Jul 28 17:15:48.438069 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:48.438038 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v72wr"] Jul 28 17:15:48.440190 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:15:48.440155 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad0332c7_5823_4e9a_9b43_ab4cacbc8449.slice/crio-f35350de4a928b4aa3dcf1da06e3623c851fb3b20daa3876f9d77133339697be WatchSource:0}: Error finding container f35350de4a928b4aa3dcf1da06e3623c851fb3b20daa3876f9d77133339697be: Status 404 returned error can't find the container with id f35350de4a928b4aa3dcf1da06e3623c851fb3b20daa3876f9d77133339697be Jul 28 17:15:49.281830 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:49.281788 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4z5f5" event={"ID":"776d3861-f435-4f91-b0d6-b4d3a1124820","Type":"ContainerStarted","Data":"f4b025f0005464b45a7bfce7780e2e806df7c16279770121ac9c5bed865ac33d"} Jul 28 17:15:49.282054 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:49.282033 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4z5f5" Jul 28 17:15:49.283126 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:49.283092 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v72wr" event={"ID":"ad0332c7-5823-4e9a-9b43-ab4cacbc8449","Type":"ContainerStarted","Data":"f35350de4a928b4aa3dcf1da06e3623c851fb3b20daa3876f9d77133339697be"} Jul 28 17:15:49.285073 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:49.285051 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sn2x6" event={"ID":"03a90820-6e21-44c0-be21-b7f42bbd6837","Type":"ContainerStarted","Data":"c6eae26eed517e898296e070237ac5a7b5ea0453baca635bc6a4b15b640fa473"} Jul 28 17:15:49.285178 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:49.285104 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sn2x6" Jul 28 17:15:49.304457 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:49.304406 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4z5f5" podStartSLOduration=2.467377923 podStartE2EDuration="6.304392006s" totalImagesPullingTime="3.837014083s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:15:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:15:49.301619364 +0000 UTC m=+447.910550733" watchObservedRunningTime="2026-07-28 17:15:49.304392006 +0000 UTC m=+447.913323376" Jul 28 17:15:49.325164 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:49.325101 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sn2x6" podStartSLOduration=2.483876807 podStartE2EDuration="8.325087175s" totalImagesPullingTime="5.841210368s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:15:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:15:49.323295753 +0000 UTC m=+447.932227123" watchObservedRunningTime="2026-07-28 17:15:49.325087175 +0000 UTC m=+447.934018547" Jul 28 17:15:50.291616 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:50.291581 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v72wr" event={"ID":"ad0332c7-5823-4e9a-9b43-ab4cacbc8449","Type":"ContainerStarted","Data":"4b6feae72bbd1dedcad30e5bc0e6eb84f34b52ece00f76d84b516453b92911d0"} Jul 28 17:15:50.292081 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:50.291892 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v72wr" Jul 28 17:15:50.322155 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:50.322086 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v72wr" podStartSLOduration=2.678493686 podStartE2EDuration="4.32207111s" totalImagesPullingTime="1.643577424s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:15:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:15:50.321371624 +0000 UTC m=+448.930303014" watchObservedRunningTime="2026-07-28 17:15:50.32207111 +0000 UTC m=+448.931002479" Jul 28 17:15:51.234060 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:15:51.234027 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wvxrn" Jul 28 17:16:00.293732 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:00.293703 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sn2x6" Jul 28 17:16:00.294525 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:00.294506 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4z5f5" Jul 28 17:16:01.299253 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:01.299224 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v72wr" Jul 28 17:16:01.963038 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:01.963005 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-xv7zq"] Jul 28 17:16:01.966450 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:01.966434 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-xv7zq" Jul 28 17:16:01.983430 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:01.983399 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-xv7zq"] Jul 28 17:16:02.036798 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.036763 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc472be2-1ebd-4845-a457-62de884aee22-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-xv7zq\" (UID: \"dc472be2-1ebd-4845-a457-62de884aee22\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-xv7zq" Jul 28 17:16:02.036798 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.036803 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xljrf\" (UniqueName: \"kubernetes.io/projected/dc472be2-1ebd-4845-a457-62de884aee22-kube-api-access-xljrf\") pod \"kuadrant-operator-controller-manager-55467d9f7-xv7zq\" (UID: \"dc472be2-1ebd-4845-a457-62de884aee22\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-xv7zq" Jul 28 17:16:02.137453 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.137414 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc472be2-1ebd-4845-a457-62de884aee22-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-xv7zq\" (UID: \"dc472be2-1ebd-4845-a457-62de884aee22\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-xv7zq" Jul 28 17:16:02.137453 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.137453 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xljrf\" (UniqueName: \"kubernetes.io/projected/dc472be2-1ebd-4845-a457-62de884aee22-kube-api-access-xljrf\") pod \"kuadrant-operator-controller-manager-55467d9f7-xv7zq\" (UID: \"dc472be2-1ebd-4845-a457-62de884aee22\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-xv7zq" Jul 28 17:16:02.137828 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.137809 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc472be2-1ebd-4845-a457-62de884aee22-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-xv7zq\" (UID: \"dc472be2-1ebd-4845-a457-62de884aee22\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-xv7zq" Jul 28 17:16:02.153414 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.153389 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xljrf\" (UniqueName: \"kubernetes.io/projected/dc472be2-1ebd-4845-a457-62de884aee22-kube-api-access-xljrf\") pod \"kuadrant-operator-controller-manager-55467d9f7-xv7zq\" (UID: \"dc472be2-1ebd-4845-a457-62de884aee22\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-xv7zq" Jul 28 17:16:02.234220 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.234131 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-xv7zq"] Jul 28 17:16:02.234442 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.234429 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-xv7zq" Jul 28 17:16:02.248642 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.248607 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-xv7zq"] Jul 28 17:16:02.268752 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.268722 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sn2x6"] Jul 28 17:16:02.268944 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.268920 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sn2x6" podUID="03a90820-6e21-44c0-be21-b7f42bbd6837" containerName="manager" containerID="cri-o://c6eae26eed517e898296e070237ac5a7b5ea0453baca635bc6a4b15b640fa473" gracePeriod=2 Jul 28 17:16:02.284402 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.284372 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sn2x6"] Jul 28 17:16:02.313072 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.313020 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v72wr"] Jul 28 17:16:02.313708 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.313254 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v72wr" podUID="ad0332c7-5823-4e9a-9b43-ab4cacbc8449" containerName="manager" containerID="cri-o://4b6feae72bbd1dedcad30e5bc0e6eb84f34b52ece00f76d84b516453b92911d0" gracePeriod=2 Jul 28 17:16:02.315486 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.315432 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bf2j8"] Jul 28 17:16:02.315998 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.315965 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03a90820-6e21-44c0-be21-b7f42bbd6837" containerName="manager" Jul 28 17:16:02.315998 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.315989 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="03a90820-6e21-44c0-be21-b7f42bbd6837" containerName="manager" Jul 28 17:16:02.316265 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.316196 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="03a90820-6e21-44c0-be21-b7f42bbd6837" containerName="manager" Jul 28 17:16:02.320076 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.320038 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v72wr"] Jul 28 17:16:02.320246 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.320212 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bf2j8" Jul 28 17:16:02.344978 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:16:02.344934 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-sn2x6\" is forbidden: User \"system:node:ip-10-0-132-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-11.ec2.internal' and this object" podUID="03a90820-6e21-44c0-be21-b7f42bbd6837" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sn2x6" Jul 28 17:16:02.348151 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:16:02.348102 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-v72wr\" is forbidden: User \"system:node:ip-10-0-132-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-11.ec2.internal' and this object" podUID="ad0332c7-5823-4e9a-9b43-ab4cacbc8449" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v72wr" Jul 28 17:16:02.351012 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:16:02.350979 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-sn2x6\" is forbidden: User \"system:node:ip-10-0-132-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-11.ec2.internal' and this object" podUID="03a90820-6e21-44c0-be21-b7f42bbd6837" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sn2x6" Jul 28 17:16:02.353463 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.353439 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dndrl"] Jul 28 17:16:02.353792 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.353777 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad0332c7-5823-4e9a-9b43-ab4cacbc8449" containerName="manager" Jul 28 17:16:02.353792 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.353790 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad0332c7-5823-4e9a-9b43-ab4cacbc8449" containerName="manager" Jul 28 17:16:02.353934 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.353869 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad0332c7-5823-4e9a-9b43-ab4cacbc8449" containerName="manager" Jul 28 17:16:02.354747 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:16:02.354722 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-v72wr\" is forbidden: User \"system:node:ip-10-0-132-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-11.ec2.internal' and this object" podUID="ad0332c7-5823-4e9a-9b43-ab4cacbc8449" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v72wr" Jul 28 17:16:02.356805 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.356783 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dndrl" Jul 28 17:16:02.386056 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.386021 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bf2j8"] Jul 28 17:16:02.438054 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:16:02.438003 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-sn2x6\" is forbidden: User \"system:node:ip-10-0-132-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-11.ec2.internal' and this object" podUID="03a90820-6e21-44c0-be21-b7f42bbd6837" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sn2x6" Jul 28 17:16:02.440409 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.440380 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xc22\" (UniqueName: \"kubernetes.io/projected/a59b4cb4-7ffa-4bd9-a66f-d2a2969780bb-kube-api-access-8xc22\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dndrl\" (UID: \"a59b4cb4-7ffa-4bd9-a66f-d2a2969780bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dndrl" Jul 28 17:16:02.440558 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.440438 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a59b4cb4-7ffa-4bd9-a66f-d2a2969780bb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dndrl\" (UID: \"a59b4cb4-7ffa-4bd9-a66f-d2a2969780bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dndrl" Jul 28 17:16:02.440558 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.440477 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/41ee774c-35fd-4df9-91a8-08cdd934f1ef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-bf2j8\" (UID: \"41ee774c-35fd-4df9-91a8-08cdd934f1ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bf2j8" Jul 28 17:16:02.440558 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.440533 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mprgc\" (UniqueName: \"kubernetes.io/projected/41ee774c-35fd-4df9-91a8-08cdd934f1ef-kube-api-access-mprgc\") pod \"kuadrant-operator-controller-manager-55467d9f7-bf2j8\" (UID: \"41ee774c-35fd-4df9-91a8-08cdd934f1ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bf2j8" Jul 28 17:16:02.449810 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.449764 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dndrl"] Jul 28 17:16:02.479729 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:16:02.479685 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-v72wr\" is forbidden: User \"system:node:ip-10-0-132-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-11.ec2.internal' and this object" podUID="ad0332c7-5823-4e9a-9b43-ab4cacbc8449" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v72wr" Jul 28 17:16:02.541869 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.541834 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xc22\" (UniqueName: \"kubernetes.io/projected/a59b4cb4-7ffa-4bd9-a66f-d2a2969780bb-kube-api-access-8xc22\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dndrl\" (UID: \"a59b4cb4-7ffa-4bd9-a66f-d2a2969780bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dndrl" Jul 28 17:16:02.542065 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.541883 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a59b4cb4-7ffa-4bd9-a66f-d2a2969780bb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dndrl\" (UID: \"a59b4cb4-7ffa-4bd9-a66f-d2a2969780bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dndrl" Jul 28 17:16:02.542065 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.541915 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/41ee774c-35fd-4df9-91a8-08cdd934f1ef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-bf2j8\" (UID: \"41ee774c-35fd-4df9-91a8-08cdd934f1ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bf2j8" Jul 28 17:16:02.542065 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.541957 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mprgc\" (UniqueName: \"kubernetes.io/projected/41ee774c-35fd-4df9-91a8-08cdd934f1ef-kube-api-access-mprgc\") pod \"kuadrant-operator-controller-manager-55467d9f7-bf2j8\" (UID: \"41ee774c-35fd-4df9-91a8-08cdd934f1ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bf2j8" Jul 28 17:16:02.542377 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.542355 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/41ee774c-35fd-4df9-91a8-08cdd934f1ef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-bf2j8\" (UID: \"41ee774c-35fd-4df9-91a8-08cdd934f1ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bf2j8" Jul 28 17:16:02.542434 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.542385 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a59b4cb4-7ffa-4bd9-a66f-d2a2969780bb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dndrl\" (UID: \"a59b4cb4-7ffa-4bd9-a66f-d2a2969780bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dndrl" Jul 28 17:16:02.558689 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.558649 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xc22\" (UniqueName: \"kubernetes.io/projected/a59b4cb4-7ffa-4bd9-a66f-d2a2969780bb-kube-api-access-8xc22\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dndrl\" (UID: \"a59b4cb4-7ffa-4bd9-a66f-d2a2969780bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dndrl" Jul 28 17:16:02.558913 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.558893 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mprgc\" (UniqueName: \"kubernetes.io/projected/41ee774c-35fd-4df9-91a8-08cdd934f1ef-kube-api-access-mprgc\") pod \"kuadrant-operator-controller-manager-55467d9f7-bf2j8\" (UID: \"41ee774c-35fd-4df9-91a8-08cdd934f1ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bf2j8" Jul 28 17:16:02.562612 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.562583 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v72wr" Jul 28 17:16:02.565857 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:16:02.565832 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-sn2x6\" is forbidden: User \"system:node:ip-10-0-132-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-11.ec2.internal' and this object" podUID="03a90820-6e21-44c0-be21-b7f42bbd6837" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sn2x6" Jul 28 17:16:02.570427 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:16:02.570401 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-v72wr\" is forbidden: User \"system:node:ip-10-0-132-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-11.ec2.internal' and this object" podUID="ad0332c7-5823-4e9a-9b43-ab4cacbc8449" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v72wr" Jul 28 17:16:02.571351 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.571336 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sn2x6" Jul 28 17:16:02.573810 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:16:02.573785 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-v72wr\" is forbidden: User \"system:node:ip-10-0-132-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-11.ec2.internal' and this object" podUID="ad0332c7-5823-4e9a-9b43-ab4cacbc8449" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v72wr" Jul 28 17:16:02.577354 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:16:02.577332 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-sn2x6\" is forbidden: User \"system:node:ip-10-0-132-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-11.ec2.internal' and this object" podUID="03a90820-6e21-44c0-be21-b7f42bbd6837" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sn2x6" Jul 28 17:16:02.642882 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.642846 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/03a90820-6e21-44c0-be21-b7f42bbd6837-extensions-socket-volume\") pod \"03a90820-6e21-44c0-be21-b7f42bbd6837\" (UID: \"03a90820-6e21-44c0-be21-b7f42bbd6837\") " Jul 28 17:16:02.643019 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.642902 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8tjd\" (UniqueName: \"kubernetes.io/projected/03a90820-6e21-44c0-be21-b7f42bbd6837-kube-api-access-k8tjd\") pod \"03a90820-6e21-44c0-be21-b7f42bbd6837\" (UID: \"03a90820-6e21-44c0-be21-b7f42bbd6837\") " Jul 28 17:16:02.643019 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.642940 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9tcd\" (UniqueName: \"kubernetes.io/projected/ad0332c7-5823-4e9a-9b43-ab4cacbc8449-kube-api-access-r9tcd\") pod \"ad0332c7-5823-4e9a-9b43-ab4cacbc8449\" (UID: \"ad0332c7-5823-4e9a-9b43-ab4cacbc8449\") " Jul 28 17:16:02.643471 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.643437 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03a90820-6e21-44c0-be21-b7f42bbd6837-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "03a90820-6e21-44c0-be21-b7f42bbd6837" (UID: "03a90820-6e21-44c0-be21-b7f42bbd6837"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:16:02.645044 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.645018 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad0332c7-5823-4e9a-9b43-ab4cacbc8449-kube-api-access-r9tcd" (OuterVolumeSpecName: "kube-api-access-r9tcd") pod "ad0332c7-5823-4e9a-9b43-ab4cacbc8449" (UID: "ad0332c7-5823-4e9a-9b43-ab4cacbc8449"). InnerVolumeSpecName "kube-api-access-r9tcd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:16:02.645148 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.645060 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03a90820-6e21-44c0-be21-b7f42bbd6837-kube-api-access-k8tjd" (OuterVolumeSpecName: "kube-api-access-k8tjd") pod "03a90820-6e21-44c0-be21-b7f42bbd6837" (UID: "03a90820-6e21-44c0-be21-b7f42bbd6837"). InnerVolumeSpecName "kube-api-access-k8tjd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:16:02.706209 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.706175 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bf2j8" Jul 28 17:16:02.712275 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.712244 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dndrl" Jul 28 17:16:02.743628 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.743592 2570 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/03a90820-6e21-44c0-be21-b7f42bbd6837-extensions-socket-volume\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:16:02.743628 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.743621 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8tjd\" (UniqueName: \"kubernetes.io/projected/03a90820-6e21-44c0-be21-b7f42bbd6837-kube-api-access-k8tjd\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:16:02.743628 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.743631 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9tcd\" (UniqueName: \"kubernetes.io/projected/ad0332c7-5823-4e9a-9b43-ab4cacbc8449-kube-api-access-r9tcd\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:16:02.875486 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.875459 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bf2j8"] Jul 28 17:16:02.878618 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:16:02.878595 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41ee774c_35fd_4df9_91a8_08cdd934f1ef.slice/crio-34dcfb6a7188b6afec5bca4d8d7fe48ea3b1eb699189ca6715b4bc1135352d81 WatchSource:0}: Error finding container 34dcfb6a7188b6afec5bca4d8d7fe48ea3b1eb699189ca6715b4bc1135352d81: Status 404 returned error can't find the container with id 34dcfb6a7188b6afec5bca4d8d7fe48ea3b1eb699189ca6715b4bc1135352d81 Jul 28 17:16:02.934340 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:02.934315 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dndrl"] Jul 28 17:16:03.356467 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:03.356435 2570 generic.go:356] "Generic (PLEG): container finished" podID="ad0332c7-5823-4e9a-9b43-ab4cacbc8449" containerID="4b6feae72bbd1dedcad30e5bc0e6eb84f34b52ece00f76d84b516453b92911d0" exitCode=0 Jul 28 17:16:03.356920 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:03.356495 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v72wr" Jul 28 17:16:03.356920 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:03.356540 2570 scope.go:117] "RemoveContainer" containerID="4b6feae72bbd1dedcad30e5bc0e6eb84f34b52ece00f76d84b516453b92911d0" Jul 28 17:16:03.358287 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:03.358261 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dndrl" event={"ID":"a59b4cb4-7ffa-4bd9-a66f-d2a2969780bb","Type":"ContainerStarted","Data":"a7adb58386d488d1ae1a1cc2c89629b9614ac5c5c59c0cbbbee17c9e8b6b1a65"} Jul 28 17:16:03.358395 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:03.358319 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dndrl" Jul 28 17:16:03.358395 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:03.358343 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dndrl" event={"ID":"a59b4cb4-7ffa-4bd9-a66f-d2a2969780bb","Type":"ContainerStarted","Data":"852ae745566f5d4079883b84e081d174da50c32f8eb52c225a022086e59f77a2"} Jul 28 17:16:03.359754 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:03.359729 2570 generic.go:356] "Generic (PLEG): container finished" podID="03a90820-6e21-44c0-be21-b7f42bbd6837" containerID="c6eae26eed517e898296e070237ac5a7b5ea0453baca635bc6a4b15b640fa473" exitCode=0 Jul 28 17:16:03.359865 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:03.359776 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sn2x6" Jul 28 17:16:03.359996 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:16:03.359974 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-sn2x6\" is forbidden: User \"system:node:ip-10-0-132-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-11.ec2.internal' and this object" podUID="03a90820-6e21-44c0-be21-b7f42bbd6837" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sn2x6" Jul 28 17:16:03.361734 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:03.361702 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bf2j8" event={"ID":"41ee774c-35fd-4df9-91a8-08cdd934f1ef","Type":"ContainerStarted","Data":"ad44dc133b34f1f537257c3b055c72025c6d18f2aa30c4770c584abc98ff9b36"} Jul 28 17:16:03.361833 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:03.361743 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bf2j8" event={"ID":"41ee774c-35fd-4df9-91a8-08cdd934f1ef","Type":"ContainerStarted","Data":"34dcfb6a7188b6afec5bca4d8d7fe48ea3b1eb699189ca6715b4bc1135352d81"} Jul 28 17:16:03.361833 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:03.361763 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bf2j8" Jul 28 17:16:03.363942 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:16:03.363915 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-v72wr\" is forbidden: User \"system:node:ip-10-0-132-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-11.ec2.internal' and this object" podUID="ad0332c7-5823-4e9a-9b43-ab4cacbc8449" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v72wr" Jul 28 17:16:03.367162 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:03.367146 2570 scope.go:117] "RemoveContainer" containerID="4b6feae72bbd1dedcad30e5bc0e6eb84f34b52ece00f76d84b516453b92911d0" Jul 28 17:16:03.367465 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:16:03.367447 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b6feae72bbd1dedcad30e5bc0e6eb84f34b52ece00f76d84b516453b92911d0\": container with ID starting with 4b6feae72bbd1dedcad30e5bc0e6eb84f34b52ece00f76d84b516453b92911d0 not found: ID does not exist" containerID="4b6feae72bbd1dedcad30e5bc0e6eb84f34b52ece00f76d84b516453b92911d0" Jul 28 17:16:03.367507 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:03.367475 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b6feae72bbd1dedcad30e5bc0e6eb84f34b52ece00f76d84b516453b92911d0"} err="failed to get container status \"4b6feae72bbd1dedcad30e5bc0e6eb84f34b52ece00f76d84b516453b92911d0\": rpc error: code = NotFound desc = could not find container \"4b6feae72bbd1dedcad30e5bc0e6eb84f34b52ece00f76d84b516453b92911d0\": container with ID starting with 4b6feae72bbd1dedcad30e5bc0e6eb84f34b52ece00f76d84b516453b92911d0 not found: ID does not exist" Jul 28 17:16:03.367507 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:03.367499 2570 scope.go:117] "RemoveContainer" containerID="c6eae26eed517e898296e070237ac5a7b5ea0453baca635bc6a4b15b640fa473" Jul 28 17:16:03.376101 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:03.376080 2570 scope.go:117] "RemoveContainer" containerID="c6eae26eed517e898296e070237ac5a7b5ea0453baca635bc6a4b15b640fa473" Jul 28 17:16:03.376429 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:16:03.376407 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6eae26eed517e898296e070237ac5a7b5ea0453baca635bc6a4b15b640fa473\": container with ID starting with c6eae26eed517e898296e070237ac5a7b5ea0453baca635bc6a4b15b640fa473 not found: ID does not exist" containerID="c6eae26eed517e898296e070237ac5a7b5ea0453baca635bc6a4b15b640fa473" Jul 28 17:16:03.376538 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:03.376433 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6eae26eed517e898296e070237ac5a7b5ea0453baca635bc6a4b15b640fa473"} err="failed to get container status \"c6eae26eed517e898296e070237ac5a7b5ea0453baca635bc6a4b15b640fa473\": rpc error: code = NotFound desc = could not find container \"c6eae26eed517e898296e070237ac5a7b5ea0453baca635bc6a4b15b640fa473\": container with ID starting with c6eae26eed517e898296e070237ac5a7b5ea0453baca635bc6a4b15b640fa473 not found: ID does not exist" Jul 28 17:16:03.394880 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:03.394832 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dndrl" podStartSLOduration=1.394814746 podStartE2EDuration="1.394814746s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:16:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:16:03.392947359 +0000 UTC m=+462.001878729" watchObservedRunningTime="2026-07-28 17:16:03.394814746 +0000 UTC m=+462.003746120" Jul 28 17:16:03.395861 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:16:03.395832 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-v72wr\" is forbidden: User \"system:node:ip-10-0-132-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-11.ec2.internal' and this object" podUID="ad0332c7-5823-4e9a-9b43-ab4cacbc8449" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v72wr" Jul 28 17:16:03.449188 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:03.449056 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bf2j8" podStartSLOduration=1.449041233 podStartE2EDuration="1.449041233s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:16:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:16:03.447923165 +0000 UTC m=+462.056854538" watchObservedRunningTime="2026-07-28 17:16:03.449041233 +0000 UTC m=+462.057972602" Jul 28 17:16:03.450623 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:16:03.450591 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-sn2x6\" is forbidden: User \"system:node:ip-10-0-132-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-11.ec2.internal' and this object" podUID="03a90820-6e21-44c0-be21-b7f42bbd6837" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sn2x6" Jul 28 17:16:03.453082 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:16:03.453061 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-v72wr\" is forbidden: User \"system:node:ip-10-0-132-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-11.ec2.internal' and this object" podUID="ad0332c7-5823-4e9a-9b43-ab4cacbc8449" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v72wr" Jul 28 17:16:03.455549 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:16:03.455524 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-sn2x6\" is forbidden: User \"system:node:ip-10-0-132-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-11.ec2.internal' and this object" podUID="03a90820-6e21-44c0-be21-b7f42bbd6837" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sn2x6" Jul 28 17:16:04.120211 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:04.120179 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03a90820-6e21-44c0-be21-b7f42bbd6837" path="/var/lib/kubelet/pods/03a90820-6e21-44c0-be21-b7f42bbd6837/volumes" Jul 28 17:16:04.120569 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:04.120555 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad0332c7-5823-4e9a-9b43-ab4cacbc8449" path="/var/lib/kubelet/pods/ad0332c7-5823-4e9a-9b43-ab4cacbc8449/volumes" Jul 28 17:16:04.787376 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:16:04.787332 2570 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-xv7zq_kuadrant-system_dc472be2-1ebd-4845-a457-62de884aee22_0(b420531fc7253486a1ce16f3dd3462ef0784da2992a1b238c3ea2d97adf50591): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-xv7zq to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" Jul 28 17:16:04.787639 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:16:04.787398 2570 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-xv7zq_kuadrant-system_dc472be2-1ebd-4845-a457-62de884aee22_0(b420531fc7253486a1ce16f3dd3462ef0784da2992a1b238c3ea2d97adf50591): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-xv7zq to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-xv7zq" Jul 28 17:16:05.375345 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:05.375311 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-xv7zq" Jul 28 17:16:05.383354 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:05.383332 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-xv7zq" Jul 28 17:16:05.386322 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:16:05.386300 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-xv7zq\" is forbidden: User \"system:node:ip-10-0-132-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-11.ec2.internal' and this object" podUID="dc472be2-1ebd-4845-a457-62de884aee22" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-xv7zq" Jul 28 17:16:05.465877 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:05.465842 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xljrf\" (UniqueName: \"kubernetes.io/projected/dc472be2-1ebd-4845-a457-62de884aee22-kube-api-access-xljrf\") pod \"dc472be2-1ebd-4845-a457-62de884aee22\" (UID: \"dc472be2-1ebd-4845-a457-62de884aee22\") " Jul 28 17:16:05.466073 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:05.465925 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc472be2-1ebd-4845-a457-62de884aee22-extensions-socket-volume\") pod \"dc472be2-1ebd-4845-a457-62de884aee22\" (UID: \"dc472be2-1ebd-4845-a457-62de884aee22\") " Jul 28 17:16:05.466240 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:05.466210 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc472be2-1ebd-4845-a457-62de884aee22-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "dc472be2-1ebd-4845-a457-62de884aee22" (UID: "dc472be2-1ebd-4845-a457-62de884aee22"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:16:05.468013 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:05.467985 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc472be2-1ebd-4845-a457-62de884aee22-kube-api-access-xljrf" (OuterVolumeSpecName: "kube-api-access-xljrf") pod "dc472be2-1ebd-4845-a457-62de884aee22" (UID: "dc472be2-1ebd-4845-a457-62de884aee22"). InnerVolumeSpecName "kube-api-access-xljrf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:16:05.566719 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:05.566678 2570 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc472be2-1ebd-4845-a457-62de884aee22-extensions-socket-volume\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:16:05.566719 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:05.566712 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xljrf\" (UniqueName: \"kubernetes.io/projected/dc472be2-1ebd-4845-a457-62de884aee22-kube-api-access-xljrf\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:16:06.119710 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:06.119678 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc472be2-1ebd-4845-a457-62de884aee22" path="/var/lib/kubelet/pods/dc472be2-1ebd-4845-a457-62de884aee22/volumes" Jul 28 17:16:06.380233 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:06.380157 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-xv7zq" Jul 28 17:16:06.386723 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:16:06.386696 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-xv7zq\" is forbidden: User \"system:node:ip-10-0-132-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-11.ec2.internal' and this object" podUID="dc472be2-1ebd-4845-a457-62de884aee22" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-xv7zq" Jul 28 17:16:14.371968 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:14.371936 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bf2j8" Jul 28 17:16:14.372452 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:14.372007 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dndrl" Jul 28 17:16:14.478598 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:14.478565 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dndrl"] Jul 28 17:16:14.478820 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:14.478797 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dndrl" podUID="a59b4cb4-7ffa-4bd9-a66f-d2a2969780bb" containerName="manager" containerID="cri-o://a7adb58386d488d1ae1a1cc2c89629b9614ac5c5c59c0cbbbee17c9e8b6b1a65" gracePeriod=10 Jul 28 17:16:14.734262 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:14.734238 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dndrl" Jul 28 17:16:14.740431 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:14.740406 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvgcx"] Jul 28 17:16:14.740746 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:14.740735 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a59b4cb4-7ffa-4bd9-a66f-d2a2969780bb" containerName="manager" Jul 28 17:16:14.740787 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:14.740747 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a59b4cb4-7ffa-4bd9-a66f-d2a2969780bb" containerName="manager" Jul 28 17:16:14.740820 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:14.740816 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="a59b4cb4-7ffa-4bd9-a66f-d2a2969780bb" containerName="manager" Jul 28 17:16:14.743606 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:14.743592 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvgcx" Jul 28 17:16:14.776027 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:14.775994 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvgcx"] Jul 28 17:16:14.849756 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:14.849729 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8xc22\" (UniqueName: \"kubernetes.io/projected/a59b4cb4-7ffa-4bd9-a66f-d2a2969780bb-kube-api-access-8xc22\") pod \"a59b4cb4-7ffa-4bd9-a66f-d2a2969780bb\" (UID: \"a59b4cb4-7ffa-4bd9-a66f-d2a2969780bb\") " Jul 28 17:16:14.849873 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:14.849777 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a59b4cb4-7ffa-4bd9-a66f-d2a2969780bb-extensions-socket-volume\") pod \"a59b4cb4-7ffa-4bd9-a66f-d2a2969780bb\" (UID: \"a59b4cb4-7ffa-4bd9-a66f-d2a2969780bb\") " Jul 28 17:16:14.849953 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:14.849934 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98ctn\" (UniqueName: \"kubernetes.io/projected/5988cae6-f955-4969-8996-1b9c663e596e-kube-api-access-98ctn\") pod \"kuadrant-operator-controller-manager-86878c769f-rvgcx\" (UID: \"5988cae6-f955-4969-8996-1b9c663e596e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvgcx" Jul 28 17:16:14.850027 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:14.850012 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5988cae6-f955-4969-8996-1b9c663e596e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rvgcx\" (UID: \"5988cae6-f955-4969-8996-1b9c663e596e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvgcx" Jul 28 17:16:14.850214 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:14.850195 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a59b4cb4-7ffa-4bd9-a66f-d2a2969780bb-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a59b4cb4-7ffa-4bd9-a66f-d2a2969780bb" (UID: "a59b4cb4-7ffa-4bd9-a66f-d2a2969780bb"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:16:14.851739 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:14.851714 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a59b4cb4-7ffa-4bd9-a66f-d2a2969780bb-kube-api-access-8xc22" (OuterVolumeSpecName: "kube-api-access-8xc22") pod "a59b4cb4-7ffa-4bd9-a66f-d2a2969780bb" (UID: "a59b4cb4-7ffa-4bd9-a66f-d2a2969780bb"). InnerVolumeSpecName "kube-api-access-8xc22". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:16:14.951047 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:14.950964 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98ctn\" (UniqueName: \"kubernetes.io/projected/5988cae6-f955-4969-8996-1b9c663e596e-kube-api-access-98ctn\") pod \"kuadrant-operator-controller-manager-86878c769f-rvgcx\" (UID: \"5988cae6-f955-4969-8996-1b9c663e596e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvgcx" Jul 28 17:16:14.951047 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:14.951022 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5988cae6-f955-4969-8996-1b9c663e596e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rvgcx\" (UID: \"5988cae6-f955-4969-8996-1b9c663e596e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvgcx" Jul 28 17:16:14.951291 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:14.951065 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xc22\" (UniqueName: \"kubernetes.io/projected/a59b4cb4-7ffa-4bd9-a66f-d2a2969780bb-kube-api-access-8xc22\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:16:14.951291 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:14.951081 2570 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a59b4cb4-7ffa-4bd9-a66f-d2a2969780bb-extensions-socket-volume\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:16:14.951424 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:14.951400 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5988cae6-f955-4969-8996-1b9c663e596e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rvgcx\" (UID: \"5988cae6-f955-4969-8996-1b9c663e596e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvgcx" Jul 28 17:16:14.973490 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:14.973455 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98ctn\" (UniqueName: \"kubernetes.io/projected/5988cae6-f955-4969-8996-1b9c663e596e-kube-api-access-98ctn\") pod \"kuadrant-operator-controller-manager-86878c769f-rvgcx\" (UID: \"5988cae6-f955-4969-8996-1b9c663e596e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvgcx" Jul 28 17:16:15.053101 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:15.053069 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvgcx" Jul 28 17:16:15.210074 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:15.210051 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvgcx"] Jul 28 17:16:15.213385 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:16:15.213357 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5988cae6_f955_4969_8996_1b9c663e596e.slice/crio-74b3524561e20dd7bf6407a1f724baec6fc539fb6b200bf195e17f7b9dcba0f7 WatchSource:0}: Error finding container 74b3524561e20dd7bf6407a1f724baec6fc539fb6b200bf195e17f7b9dcba0f7: Status 404 returned error can't find the container with id 74b3524561e20dd7bf6407a1f724baec6fc539fb6b200bf195e17f7b9dcba0f7 Jul 28 17:16:15.434657 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:15.434614 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvgcx" event={"ID":"5988cae6-f955-4969-8996-1b9c663e596e","Type":"ContainerStarted","Data":"f85e987e5e69275eb7a62ca48f99c0be295c47132cc43cd54647dadd5ad92226"} Jul 28 17:16:15.435104 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:15.434687 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvgcx" Jul 28 17:16:15.435104 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:15.434714 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvgcx" event={"ID":"5988cae6-f955-4969-8996-1b9c663e596e","Type":"ContainerStarted","Data":"74b3524561e20dd7bf6407a1f724baec6fc539fb6b200bf195e17f7b9dcba0f7"} Jul 28 17:16:15.436002 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:15.435980 2570 generic.go:356] "Generic (PLEG): container finished" podID="a59b4cb4-7ffa-4bd9-a66f-d2a2969780bb" containerID="a7adb58386d488d1ae1a1cc2c89629b9614ac5c5c59c0cbbbee17c9e8b6b1a65" exitCode=0 Jul 28 17:16:15.436135 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:15.436007 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dndrl" event={"ID":"a59b4cb4-7ffa-4bd9-a66f-d2a2969780bb","Type":"ContainerDied","Data":"a7adb58386d488d1ae1a1cc2c89629b9614ac5c5c59c0cbbbee17c9e8b6b1a65"} Jul 28 17:16:15.436135 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:15.436030 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dndrl" Jul 28 17:16:15.436135 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:15.436036 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dndrl" event={"ID":"a59b4cb4-7ffa-4bd9-a66f-d2a2969780bb","Type":"ContainerDied","Data":"852ae745566f5d4079883b84e081d174da50c32f8eb52c225a022086e59f77a2"} Jul 28 17:16:15.436135 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:15.436068 2570 scope.go:117] "RemoveContainer" containerID="a7adb58386d488d1ae1a1cc2c89629b9614ac5c5c59c0cbbbee17c9e8b6b1a65" Jul 28 17:16:15.445626 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:15.445608 2570 scope.go:117] "RemoveContainer" containerID="a7adb58386d488d1ae1a1cc2c89629b9614ac5c5c59c0cbbbee17c9e8b6b1a65" Jul 28 17:16:15.445859 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:16:15.445842 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7adb58386d488d1ae1a1cc2c89629b9614ac5c5c59c0cbbbee17c9e8b6b1a65\": container with ID starting with a7adb58386d488d1ae1a1cc2c89629b9614ac5c5c59c0cbbbee17c9e8b6b1a65 not found: ID does not exist" containerID="a7adb58386d488d1ae1a1cc2c89629b9614ac5c5c59c0cbbbee17c9e8b6b1a65" Jul 28 17:16:15.445911 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:15.445869 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7adb58386d488d1ae1a1cc2c89629b9614ac5c5c59c0cbbbee17c9e8b6b1a65"} err="failed to get container status \"a7adb58386d488d1ae1a1cc2c89629b9614ac5c5c59c0cbbbee17c9e8b6b1a65\": rpc error: code = NotFound desc = could not find container \"a7adb58386d488d1ae1a1cc2c89629b9614ac5c5c59c0cbbbee17c9e8b6b1a65\": container with ID starting with a7adb58386d488d1ae1a1cc2c89629b9614ac5c5c59c0cbbbee17c9e8b6b1a65 not found: ID does not exist" Jul 28 17:16:15.460403 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:15.460318 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvgcx" podStartSLOduration=1.460306002 podStartE2EDuration="1.460306002s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:16:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:16:15.458026805 +0000 UTC m=+474.066958176" watchObservedRunningTime="2026-07-28 17:16:15.460306002 +0000 UTC m=+474.069237373" Jul 28 17:16:15.478750 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:15.478723 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dndrl"] Jul 28 17:16:15.480700 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:15.480679 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dndrl"] Jul 28 17:16:16.120637 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:16.120604 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a59b4cb4-7ffa-4bd9-a66f-d2a2969780bb" path="/var/lib/kubelet/pods/a59b4cb4-7ffa-4bd9-a66f-d2a2969780bb/volumes" Jul 28 17:16:26.444839 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:26.444806 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvgcx" Jul 28 17:16:26.503041 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:26.503009 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bf2j8"] Jul 28 17:16:26.503272 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:26.503251 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bf2j8" podUID="41ee774c-35fd-4df9-91a8-08cdd934f1ef" containerName="manager" containerID="cri-o://ad44dc133b34f1f537257c3b055c72025c6d18f2aa30c4770c584abc98ff9b36" gracePeriod=10 Jul 28 17:16:26.759547 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:26.759524 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bf2j8" Jul 28 17:16:26.954810 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:26.954777 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mprgc\" (UniqueName: \"kubernetes.io/projected/41ee774c-35fd-4df9-91a8-08cdd934f1ef-kube-api-access-mprgc\") pod \"41ee774c-35fd-4df9-91a8-08cdd934f1ef\" (UID: \"41ee774c-35fd-4df9-91a8-08cdd934f1ef\") " Jul 28 17:16:26.954968 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:26.954826 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/41ee774c-35fd-4df9-91a8-08cdd934f1ef-extensions-socket-volume\") pod \"41ee774c-35fd-4df9-91a8-08cdd934f1ef\" (UID: \"41ee774c-35fd-4df9-91a8-08cdd934f1ef\") " Jul 28 17:16:26.955240 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:26.955217 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41ee774c-35fd-4df9-91a8-08cdd934f1ef-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "41ee774c-35fd-4df9-91a8-08cdd934f1ef" (UID: "41ee774c-35fd-4df9-91a8-08cdd934f1ef"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:16:26.956924 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:26.956899 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41ee774c-35fd-4df9-91a8-08cdd934f1ef-kube-api-access-mprgc" (OuterVolumeSpecName: "kube-api-access-mprgc") pod "41ee774c-35fd-4df9-91a8-08cdd934f1ef" (UID: "41ee774c-35fd-4df9-91a8-08cdd934f1ef"). InnerVolumeSpecName "kube-api-access-mprgc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:16:27.055787 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:27.055698 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mprgc\" (UniqueName: \"kubernetes.io/projected/41ee774c-35fd-4df9-91a8-08cdd934f1ef-kube-api-access-mprgc\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:16:27.055787 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:27.055730 2570 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/41ee774c-35fd-4df9-91a8-08cdd934f1ef-extensions-socket-volume\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:16:27.501551 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:27.501512 2570 generic.go:356] "Generic (PLEG): container finished" podID="41ee774c-35fd-4df9-91a8-08cdd934f1ef" containerID="ad44dc133b34f1f537257c3b055c72025c6d18f2aa30c4770c584abc98ff9b36" exitCode=0 Jul 28 17:16:27.501937 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:27.501557 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bf2j8" Jul 28 17:16:27.501937 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:27.501567 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bf2j8" event={"ID":"41ee774c-35fd-4df9-91a8-08cdd934f1ef","Type":"ContainerDied","Data":"ad44dc133b34f1f537257c3b055c72025c6d18f2aa30c4770c584abc98ff9b36"} Jul 28 17:16:27.501937 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:27.501618 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bf2j8" event={"ID":"41ee774c-35fd-4df9-91a8-08cdd934f1ef","Type":"ContainerDied","Data":"34dcfb6a7188b6afec5bca4d8d7fe48ea3b1eb699189ca6715b4bc1135352d81"} Jul 28 17:16:27.501937 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:27.501649 2570 scope.go:117] "RemoveContainer" containerID="ad44dc133b34f1f537257c3b055c72025c6d18f2aa30c4770c584abc98ff9b36" Jul 28 17:16:27.511241 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:27.511011 2570 scope.go:117] "RemoveContainer" containerID="ad44dc133b34f1f537257c3b055c72025c6d18f2aa30c4770c584abc98ff9b36" Jul 28 17:16:27.511336 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:16:27.511315 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad44dc133b34f1f537257c3b055c72025c6d18f2aa30c4770c584abc98ff9b36\": container with ID starting with ad44dc133b34f1f537257c3b055c72025c6d18f2aa30c4770c584abc98ff9b36 not found: ID does not exist" containerID="ad44dc133b34f1f537257c3b055c72025c6d18f2aa30c4770c584abc98ff9b36" Jul 28 17:16:27.511376 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:27.511345 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad44dc133b34f1f537257c3b055c72025c6d18f2aa30c4770c584abc98ff9b36"} err="failed to get container status \"ad44dc133b34f1f537257c3b055c72025c6d18f2aa30c4770c584abc98ff9b36\": rpc error: code = NotFound desc = could not find container \"ad44dc133b34f1f537257c3b055c72025c6d18f2aa30c4770c584abc98ff9b36\": container with ID starting with ad44dc133b34f1f537257c3b055c72025c6d18f2aa30c4770c584abc98ff9b36 not found: ID does not exist" Jul 28 17:16:27.532655 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:27.532612 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bf2j8"] Jul 28 17:16:27.535361 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:27.535339 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bf2j8"] Jul 28 17:16:28.120017 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:28.119977 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41ee774c-35fd-4df9-91a8-08cdd934f1ef" path="/var/lib/kubelet/pods/41ee774c-35fd-4df9-91a8-08cdd934f1ef/volumes" Jul 28 17:16:31.126527 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:31.126494 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw"] Jul 28 17:16:31.127011 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:31.126891 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41ee774c-35fd-4df9-91a8-08cdd934f1ef" containerName="manager" Jul 28 17:16:31.127011 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:31.126905 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="41ee774c-35fd-4df9-91a8-08cdd934f1ef" containerName="manager" Jul 28 17:16:31.127011 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:31.126965 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="41ee774c-35fd-4df9-91a8-08cdd934f1ef" containerName="manager" Jul 28 17:16:31.130991 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:31.130973 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" Jul 28 17:16:31.134004 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:31.133986 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-r2688\"" Jul 28 17:16:31.141692 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:31.141669 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw"] Jul 28 17:16:31.284964 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:31.284925 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ecd3cc49-6ce7-4db3-a0d0-797b1600822b-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vnfxw\" (UID: \"ecd3cc49-6ce7-4db3-a0d0-797b1600822b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" Jul 28 17:16:31.284964 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:31.284967 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xz4vr\" (UniqueName: \"kubernetes.io/projected/ecd3cc49-6ce7-4db3-a0d0-797b1600822b-kube-api-access-xz4vr\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vnfxw\" (UID: \"ecd3cc49-6ce7-4db3-a0d0-797b1600822b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" Jul 28 17:16:31.285222 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:31.284987 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ecd3cc49-6ce7-4db3-a0d0-797b1600822b-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vnfxw\" (UID: \"ecd3cc49-6ce7-4db3-a0d0-797b1600822b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" Jul 28 17:16:31.285222 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:31.285075 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecd3cc49-6ce7-4db3-a0d0-797b1600822b-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vnfxw\" (UID: \"ecd3cc49-6ce7-4db3-a0d0-797b1600822b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" Jul 28 17:16:31.285222 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:31.285107 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ecd3cc49-6ce7-4db3-a0d0-797b1600822b-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vnfxw\" (UID: \"ecd3cc49-6ce7-4db3-a0d0-797b1600822b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" Jul 28 17:16:31.285377 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:31.285294 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ecd3cc49-6ce7-4db3-a0d0-797b1600822b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vnfxw\" (UID: \"ecd3cc49-6ce7-4db3-a0d0-797b1600822b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" Jul 28 17:16:31.285377 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:31.285360 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ecd3cc49-6ce7-4db3-a0d0-797b1600822b-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vnfxw\" (UID: \"ecd3cc49-6ce7-4db3-a0d0-797b1600822b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" Jul 28 17:16:31.285475 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:31.285390 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ecd3cc49-6ce7-4db3-a0d0-797b1600822b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vnfxw\" (UID: \"ecd3cc49-6ce7-4db3-a0d0-797b1600822b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" Jul 28 17:16:31.285475 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:31.285442 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecd3cc49-6ce7-4db3-a0d0-797b1600822b-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vnfxw\" (UID: \"ecd3cc49-6ce7-4db3-a0d0-797b1600822b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" Jul 28 17:16:31.385954 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:31.385859 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ecd3cc49-6ce7-4db3-a0d0-797b1600822b-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vnfxw\" (UID: \"ecd3cc49-6ce7-4db3-a0d0-797b1600822b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" Jul 28 17:16:31.385954 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:31.385913 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xz4vr\" (UniqueName: \"kubernetes.io/projected/ecd3cc49-6ce7-4db3-a0d0-797b1600822b-kube-api-access-xz4vr\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vnfxw\" (UID: \"ecd3cc49-6ce7-4db3-a0d0-797b1600822b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" Jul 28 17:16:31.385954 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:31.385943 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ecd3cc49-6ce7-4db3-a0d0-797b1600822b-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vnfxw\" (UID: \"ecd3cc49-6ce7-4db3-a0d0-797b1600822b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" Jul 28 17:16:31.386271 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:31.385976 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecd3cc49-6ce7-4db3-a0d0-797b1600822b-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vnfxw\" (UID: \"ecd3cc49-6ce7-4db3-a0d0-797b1600822b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" Jul 28 17:16:31.386271 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:31.386002 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ecd3cc49-6ce7-4db3-a0d0-797b1600822b-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vnfxw\" (UID: \"ecd3cc49-6ce7-4db3-a0d0-797b1600822b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" Jul 28 17:16:31.386271 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:31.386039 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ecd3cc49-6ce7-4db3-a0d0-797b1600822b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vnfxw\" (UID: \"ecd3cc49-6ce7-4db3-a0d0-797b1600822b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" Jul 28 17:16:31.386271 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:31.386076 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ecd3cc49-6ce7-4db3-a0d0-797b1600822b-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vnfxw\" (UID: \"ecd3cc49-6ce7-4db3-a0d0-797b1600822b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" Jul 28 17:16:31.386271 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:31.386102 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ecd3cc49-6ce7-4db3-a0d0-797b1600822b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vnfxw\" (UID: \"ecd3cc49-6ce7-4db3-a0d0-797b1600822b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" Jul 28 17:16:31.386271 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:31.386176 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecd3cc49-6ce7-4db3-a0d0-797b1600822b-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vnfxw\" (UID: \"ecd3cc49-6ce7-4db3-a0d0-797b1600822b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" Jul 28 17:16:31.386600 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:31.386457 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecd3cc49-6ce7-4db3-a0d0-797b1600822b-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vnfxw\" (UID: \"ecd3cc49-6ce7-4db3-a0d0-797b1600822b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" Jul 28 17:16:31.386600 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:31.386508 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ecd3cc49-6ce7-4db3-a0d0-797b1600822b-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vnfxw\" (UID: \"ecd3cc49-6ce7-4db3-a0d0-797b1600822b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" Jul 28 17:16:31.386600 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:31.386544 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ecd3cc49-6ce7-4db3-a0d0-797b1600822b-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vnfxw\" (UID: \"ecd3cc49-6ce7-4db3-a0d0-797b1600822b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" Jul 28 17:16:31.386600 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:31.386590 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecd3cc49-6ce7-4db3-a0d0-797b1600822b-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vnfxw\" (UID: \"ecd3cc49-6ce7-4db3-a0d0-797b1600822b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" Jul 28 17:16:31.386847 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:31.386820 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ecd3cc49-6ce7-4db3-a0d0-797b1600822b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vnfxw\" (UID: \"ecd3cc49-6ce7-4db3-a0d0-797b1600822b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" Jul 28 17:16:31.388305 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:31.388286 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ecd3cc49-6ce7-4db3-a0d0-797b1600822b-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vnfxw\" (UID: \"ecd3cc49-6ce7-4db3-a0d0-797b1600822b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" Jul 28 17:16:31.388617 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:31.388602 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ecd3cc49-6ce7-4db3-a0d0-797b1600822b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vnfxw\" (UID: \"ecd3cc49-6ce7-4db3-a0d0-797b1600822b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" Jul 28 17:16:31.395475 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:31.395444 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xz4vr\" (UniqueName: \"kubernetes.io/projected/ecd3cc49-6ce7-4db3-a0d0-797b1600822b-kube-api-access-xz4vr\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vnfxw\" (UID: \"ecd3cc49-6ce7-4db3-a0d0-797b1600822b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" Jul 28 17:16:31.395718 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:31.395695 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ecd3cc49-6ce7-4db3-a0d0-797b1600822b-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vnfxw\" (UID: \"ecd3cc49-6ce7-4db3-a0d0-797b1600822b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" Jul 28 17:16:31.441653 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:31.441629 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" Jul 28 17:16:31.577759 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:31.577734 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw"] Jul 28 17:16:31.579326 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:16:31.579299 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecd3cc49_6ce7_4db3_a0d0_797b1600822b.slice/crio-a2f3b3ea3da7456329dc7c8bf3879c1608f9ff08ac1c1eac684734c83406b91a WatchSource:0}: Error finding container a2f3b3ea3da7456329dc7c8bf3879c1608f9ff08ac1c1eac684734c83406b91a: Status 404 returned error can't find the container with id a2f3b3ea3da7456329dc7c8bf3879c1608f9ff08ac1c1eac684734c83406b91a Jul 28 17:16:31.581256 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:31.581229 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 17:16:31.581316 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:31.581294 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 17:16:31.581358 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:31.581320 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 17:16:32.532882 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:32.532838 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" event={"ID":"ecd3cc49-6ce7-4db3-a0d0-797b1600822b","Type":"ContainerStarted","Data":"44244b60769bbb4707ea896febc387ec5f84740b1f30208311ec5b00e79f83ee"} Jul 28 17:16:32.533285 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:32.532899 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" event={"ID":"ecd3cc49-6ce7-4db3-a0d0-797b1600822b","Type":"ContainerStarted","Data":"a2f3b3ea3da7456329dc7c8bf3879c1608f9ff08ac1c1eac684734c83406b91a"} Jul 28 17:16:32.564535 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:32.564475 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" podStartSLOduration=1.564455334 podStartE2EDuration="1.564455334s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:16:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:16:32.563310231 +0000 UTC m=+491.172241601" watchObservedRunningTime="2026-07-28 17:16:32.564455334 +0000 UTC m=+491.173386706" Jul 28 17:16:33.442289 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:33.442253 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" Jul 28 17:16:33.447198 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:33.447173 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" Jul 28 17:16:33.537663 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:33.537631 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" Jul 28 17:16:33.538678 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:33.538652 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" Jul 28 17:16:42.338014 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:42.337971 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fbxpj"] Jul 28 17:16:42.342628 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:42.342603 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-fbxpj" Jul 28 17:16:42.347962 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:42.347936 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-m4mtg\"" Jul 28 17:16:42.348101 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:42.347978 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 28 17:16:42.359549 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:42.359524 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fbxpj"] Jul 28 17:16:42.383317 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:42.383289 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2487c85c-45bb-4eea-930a-1e8493b3a1f0-config-file\") pod \"limitador-limitador-64c94fd64f-fbxpj\" (UID: \"2487c85c-45bb-4eea-930a-1e8493b3a1f0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-fbxpj" Jul 28 17:16:42.383507 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:42.383438 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kq64z\" (UniqueName: \"kubernetes.io/projected/2487c85c-45bb-4eea-930a-1e8493b3a1f0-kube-api-access-kq64z\") pod \"limitador-limitador-64c94fd64f-fbxpj\" (UID: \"2487c85c-45bb-4eea-930a-1e8493b3a1f0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-fbxpj" Jul 28 17:16:42.437574 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:42.437544 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fbxpj"] Jul 28 17:16:42.484653 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:42.484621 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kq64z\" (UniqueName: \"kubernetes.io/projected/2487c85c-45bb-4eea-930a-1e8493b3a1f0-kube-api-access-kq64z\") pod \"limitador-limitador-64c94fd64f-fbxpj\" (UID: \"2487c85c-45bb-4eea-930a-1e8493b3a1f0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-fbxpj" Jul 28 17:16:42.484809 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:42.484664 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2487c85c-45bb-4eea-930a-1e8493b3a1f0-config-file\") pod \"limitador-limitador-64c94fd64f-fbxpj\" (UID: \"2487c85c-45bb-4eea-930a-1e8493b3a1f0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-fbxpj" Jul 28 17:16:42.485331 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:42.485314 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2487c85c-45bb-4eea-930a-1e8493b3a1f0-config-file\") pod \"limitador-limitador-64c94fd64f-fbxpj\" (UID: \"2487c85c-45bb-4eea-930a-1e8493b3a1f0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-fbxpj" Jul 28 17:16:42.497696 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:42.497672 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kq64z\" (UniqueName: \"kubernetes.io/projected/2487c85c-45bb-4eea-930a-1e8493b3a1f0-kube-api-access-kq64z\") pod \"limitador-limitador-64c94fd64f-fbxpj\" (UID: \"2487c85c-45bb-4eea-930a-1e8493b3a1f0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-fbxpj" Jul 28 17:16:42.653771 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:42.653737 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-fbxpj" Jul 28 17:16:42.808731 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:42.808599 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fbxpj"] Jul 28 17:16:42.810929 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:16:42.810904 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2487c85c_45bb_4eea_930a_1e8493b3a1f0.slice/crio-a8715ade56f0cefdd8e9b1c9df34b778c17af9e13b078467c7491703ce550261 WatchSource:0}: Error finding container a8715ade56f0cefdd8e9b1c9df34b778c17af9e13b078467c7491703ce550261: Status 404 returned error can't find the container with id a8715ade56f0cefdd8e9b1c9df34b778c17af9e13b078467c7491703ce550261 Jul 28 17:16:43.353265 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:43.353226 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-g2h76"] Jul 28 17:16:43.357893 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:43.357872 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-g2h76" Jul 28 17:16:43.361893 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:43.361871 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-cnrqv\"" Jul 28 17:16:43.385638 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:43.385596 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-g2h76"] Jul 28 17:16:43.393423 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:43.393392 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xnvf\" (UniqueName: \"kubernetes.io/projected/447ccc2b-9fa7-4efa-9d76-65b933b6cd06-kube-api-access-6xnvf\") pod \"authorino-58cd955675-g2h76\" (UID: \"447ccc2b-9fa7-4efa-9d76-65b933b6cd06\") " pod="kuadrant-system/authorino-58cd955675-g2h76" Jul 28 17:16:43.494311 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:43.494274 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xnvf\" (UniqueName: \"kubernetes.io/projected/447ccc2b-9fa7-4efa-9d76-65b933b6cd06-kube-api-access-6xnvf\") pod \"authorino-58cd955675-g2h76\" (UID: \"447ccc2b-9fa7-4efa-9d76-65b933b6cd06\") " pod="kuadrant-system/authorino-58cd955675-g2h76" Jul 28 17:16:43.508753 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:43.508718 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xnvf\" (UniqueName: \"kubernetes.io/projected/447ccc2b-9fa7-4efa-9d76-65b933b6cd06-kube-api-access-6xnvf\") pod \"authorino-58cd955675-g2h76\" (UID: \"447ccc2b-9fa7-4efa-9d76-65b933b6cd06\") " pod="kuadrant-system/authorino-58cd955675-g2h76" Jul 28 17:16:43.586672 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:43.586636 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-fbxpj" event={"ID":"2487c85c-45bb-4eea-930a-1e8493b3a1f0","Type":"ContainerStarted","Data":"a8715ade56f0cefdd8e9b1c9df34b778c17af9e13b078467c7491703ce550261"} Jul 28 17:16:43.666904 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:43.666872 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-g2h76" Jul 28 17:16:43.750560 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:43.748761 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-m5tpb"] Jul 28 17:16:43.754845 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:43.754585 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-m5tpb" Jul 28 17:16:43.765620 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:43.765588 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-m5tpb"] Jul 28 17:16:43.797530 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:43.797497 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfvnl\" (UniqueName: \"kubernetes.io/projected/c57fe779-a6fd-4394-9406-7de3cafb39e5-kube-api-access-mfvnl\") pod \"authorino-75d556bd77-m5tpb\" (UID: \"c57fe779-a6fd-4394-9406-7de3cafb39e5\") " pod="kuadrant-system/authorino-75d556bd77-m5tpb" Jul 28 17:16:43.820825 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:43.820791 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-g2h76"] Jul 28 17:16:43.824820 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:16:43.824778 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod447ccc2b_9fa7_4efa_9d76_65b933b6cd06.slice/crio-ef4a9a644d05e99a7dcbfbeddb5c4403d91c3e9d9bcd03a356006feb478d8e71 WatchSource:0}: Error finding container ef4a9a644d05e99a7dcbfbeddb5c4403d91c3e9d9bcd03a356006feb478d8e71: Status 404 returned error can't find the container with id ef4a9a644d05e99a7dcbfbeddb5c4403d91c3e9d9bcd03a356006feb478d8e71 Jul 28 17:16:43.898464 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:43.898429 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfvnl\" (UniqueName: \"kubernetes.io/projected/c57fe779-a6fd-4394-9406-7de3cafb39e5-kube-api-access-mfvnl\") pod \"authorino-75d556bd77-m5tpb\" (UID: \"c57fe779-a6fd-4394-9406-7de3cafb39e5\") " pod="kuadrant-system/authorino-75d556bd77-m5tpb" Jul 28 17:16:43.910016 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:43.909960 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfvnl\" (UniqueName: \"kubernetes.io/projected/c57fe779-a6fd-4394-9406-7de3cafb39e5-kube-api-access-mfvnl\") pod \"authorino-75d556bd77-m5tpb\" (UID: \"c57fe779-a6fd-4394-9406-7de3cafb39e5\") " pod="kuadrant-system/authorino-75d556bd77-m5tpb" Jul 28 17:16:44.066963 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:44.066882 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-m5tpb" Jul 28 17:16:44.270983 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:44.270921 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-m5tpb"] Jul 28 17:16:44.270983 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:16:44.270931 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc57fe779_a6fd_4394_9406_7de3cafb39e5.slice/crio-7ad7223613ccde65a265f351383f3b69e93451619c07d7e9221bf9c9790c7cfc WatchSource:0}: Error finding container 7ad7223613ccde65a265f351383f3b69e93451619c07d7e9221bf9c9790c7cfc: Status 404 returned error can't find the container with id 7ad7223613ccde65a265f351383f3b69e93451619c07d7e9221bf9c9790c7cfc Jul 28 17:16:44.597149 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:44.597050 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-m5tpb" event={"ID":"c57fe779-a6fd-4394-9406-7de3cafb39e5","Type":"ContainerStarted","Data":"7ad7223613ccde65a265f351383f3b69e93451619c07d7e9221bf9c9790c7cfc"} Jul 28 17:16:44.599358 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:44.599285 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-g2h76" event={"ID":"447ccc2b-9fa7-4efa-9d76-65b933b6cd06","Type":"ContainerStarted","Data":"ef4a9a644d05e99a7dcbfbeddb5c4403d91c3e9d9bcd03a356006feb478d8e71"} Jul 28 17:16:48.629452 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:48.629352 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-fbxpj" event={"ID":"2487c85c-45bb-4eea-930a-1e8493b3a1f0","Type":"ContainerStarted","Data":"9af723e3b5fbf0176d07e9945f6853c8f0c6425d56c7befee835a6e81a2860f9"} Jul 28 17:16:48.629882 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:48.629458 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-fbxpj" Jul 28 17:16:48.630836 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:48.630813 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-m5tpb" event={"ID":"c57fe779-a6fd-4394-9406-7de3cafb39e5","Type":"ContainerStarted","Data":"b1b60341fac93356fff9b6f4561b83471bfefc8483ee9dfc514c8d37254c1d16"} Jul 28 17:16:48.632157 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:48.632134 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-g2h76" event={"ID":"447ccc2b-9fa7-4efa-9d76-65b933b6cd06","Type":"ContainerStarted","Data":"358a6c667053e3a77b7e27f33cda60a4db56474893393a176194393c1b6b7d07"} Jul 28 17:16:48.655361 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:48.655315 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-fbxpj" podStartSLOduration=1.085728824 podStartE2EDuration="6.65530194s" totalImagesPullingTime="5.569573116s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:16:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:16:48.654824231 +0000 UTC m=+507.263755601" watchObservedRunningTime="2026-07-28 17:16:48.65530194 +0000 UTC m=+507.264233310" Jul 28 17:16:48.686757 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:48.686713 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-m5tpb" podStartSLOduration=1.565085937 podStartE2EDuration="5.686700388s" totalImagesPullingTime="4.121614451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:16:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:16:48.686373927 +0000 UTC m=+507.295305317" watchObservedRunningTime="2026-07-28 17:16:48.686700388 +0000 UTC m=+507.295631758" Jul 28 17:16:48.727592 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:48.727545 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-g2h76" podStartSLOduration=1.1717794559999999 podStartE2EDuration="5.727529829s" totalImagesPullingTime="4.555750373s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:16:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:16:48.723332183 +0000 UTC m=+507.332263554" watchObservedRunningTime="2026-07-28 17:16:48.727529829 +0000 UTC m=+507.336461199" Jul 28 17:16:48.729063 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:48.729036 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-g2h76"] Jul 28 17:16:50.641489 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:50.641447 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-g2h76" podUID="447ccc2b-9fa7-4efa-9d76-65b933b6cd06" containerName="authorino" containerID="cri-o://358a6c667053e3a77b7e27f33cda60a4db56474893393a176194393c1b6b7d07" gracePeriod=30 Jul 28 17:16:50.895068 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:50.895009 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-g2h76" Jul 28 17:16:50.967206 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:50.967170 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6xnvf\" (UniqueName: \"kubernetes.io/projected/447ccc2b-9fa7-4efa-9d76-65b933b6cd06-kube-api-access-6xnvf\") pod \"447ccc2b-9fa7-4efa-9d76-65b933b6cd06\" (UID: \"447ccc2b-9fa7-4efa-9d76-65b933b6cd06\") " Jul 28 17:16:50.969281 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:50.969254 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/447ccc2b-9fa7-4efa-9d76-65b933b6cd06-kube-api-access-6xnvf" (OuterVolumeSpecName: "kube-api-access-6xnvf") pod "447ccc2b-9fa7-4efa-9d76-65b933b6cd06" (UID: "447ccc2b-9fa7-4efa-9d76-65b933b6cd06"). InnerVolumeSpecName "kube-api-access-6xnvf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:16:51.068073 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:51.068019 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xnvf\" (UniqueName: \"kubernetes.io/projected/447ccc2b-9fa7-4efa-9d76-65b933b6cd06-kube-api-access-6xnvf\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:16:51.647547 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:51.647514 2570 generic.go:356] "Generic (PLEG): container finished" podID="447ccc2b-9fa7-4efa-9d76-65b933b6cd06" containerID="358a6c667053e3a77b7e27f33cda60a4db56474893393a176194393c1b6b7d07" exitCode=0 Jul 28 17:16:51.647927 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:51.647558 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-g2h76" Jul 28 17:16:51.647927 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:51.647562 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-g2h76" event={"ID":"447ccc2b-9fa7-4efa-9d76-65b933b6cd06","Type":"ContainerDied","Data":"358a6c667053e3a77b7e27f33cda60a4db56474893393a176194393c1b6b7d07"} Jul 28 17:16:51.647927 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:51.647679 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-g2h76" event={"ID":"447ccc2b-9fa7-4efa-9d76-65b933b6cd06","Type":"ContainerDied","Data":"ef4a9a644d05e99a7dcbfbeddb5c4403d91c3e9d9bcd03a356006feb478d8e71"} Jul 28 17:16:51.647927 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:51.647708 2570 scope.go:117] "RemoveContainer" containerID="358a6c667053e3a77b7e27f33cda60a4db56474893393a176194393c1b6b7d07" Jul 28 17:16:51.656620 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:51.656600 2570 scope.go:117] "RemoveContainer" containerID="358a6c667053e3a77b7e27f33cda60a4db56474893393a176194393c1b6b7d07" Jul 28 17:16:51.656852 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:16:51.656833 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"358a6c667053e3a77b7e27f33cda60a4db56474893393a176194393c1b6b7d07\": container with ID starting with 358a6c667053e3a77b7e27f33cda60a4db56474893393a176194393c1b6b7d07 not found: ID does not exist" containerID="358a6c667053e3a77b7e27f33cda60a4db56474893393a176194393c1b6b7d07" Jul 28 17:16:51.656897 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:51.656860 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"358a6c667053e3a77b7e27f33cda60a4db56474893393a176194393c1b6b7d07"} err="failed to get container status \"358a6c667053e3a77b7e27f33cda60a4db56474893393a176194393c1b6b7d07\": rpc error: code = NotFound desc = could not find container \"358a6c667053e3a77b7e27f33cda60a4db56474893393a176194393c1b6b7d07\": container with ID starting with 358a6c667053e3a77b7e27f33cda60a4db56474893393a176194393c1b6b7d07 not found: ID does not exist" Jul 28 17:16:51.673418 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:51.673390 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-g2h76"] Jul 28 17:16:51.677819 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:51.677797 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-g2h76"] Jul 28 17:16:52.119228 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:52.119154 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="447ccc2b-9fa7-4efa-9d76-65b933b6cd06" path="/var/lib/kubelet/pods/447ccc2b-9fa7-4efa-9d76-65b933b6cd06/volumes" Jul 28 17:16:57.980860 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:57.980825 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fbxpj"] Jul 28 17:16:57.981253 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:57.981014 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-fbxpj" podUID="2487c85c-45bb-4eea-930a-1e8493b3a1f0" containerName="limitador" containerID="cri-o://9af723e3b5fbf0176d07e9945f6853c8f0c6425d56c7befee835a6e81a2860f9" gracePeriod=30 Jul 28 17:16:57.983188 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:57.983166 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-fbxpj" Jul 28 17:16:58.540922 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:58.540862 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-fbxpj" Jul 28 17:16:58.634973 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:58.634941 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kq64z\" (UniqueName: \"kubernetes.io/projected/2487c85c-45bb-4eea-930a-1e8493b3a1f0-kube-api-access-kq64z\") pod \"2487c85c-45bb-4eea-930a-1e8493b3a1f0\" (UID: \"2487c85c-45bb-4eea-930a-1e8493b3a1f0\") " Jul 28 17:16:58.635166 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:58.635020 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2487c85c-45bb-4eea-930a-1e8493b3a1f0-config-file\") pod \"2487c85c-45bb-4eea-930a-1e8493b3a1f0\" (UID: \"2487c85c-45bb-4eea-930a-1e8493b3a1f0\") " Jul 28 17:16:58.635422 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:58.635396 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2487c85c-45bb-4eea-930a-1e8493b3a1f0-config-file" (OuterVolumeSpecName: "config-file") pod "2487c85c-45bb-4eea-930a-1e8493b3a1f0" (UID: "2487c85c-45bb-4eea-930a-1e8493b3a1f0"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:16:58.637124 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:58.637094 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2487c85c-45bb-4eea-930a-1e8493b3a1f0-kube-api-access-kq64z" (OuterVolumeSpecName: "kube-api-access-kq64z") pod "2487c85c-45bb-4eea-930a-1e8493b3a1f0" (UID: "2487c85c-45bb-4eea-930a-1e8493b3a1f0"). InnerVolumeSpecName "kube-api-access-kq64z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:16:58.684706 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:58.684673 2570 generic.go:356] "Generic (PLEG): container finished" podID="2487c85c-45bb-4eea-930a-1e8493b3a1f0" containerID="9af723e3b5fbf0176d07e9945f6853c8f0c6425d56c7befee835a6e81a2860f9" exitCode=0 Jul 28 17:16:58.684843 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:58.684718 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-fbxpj" Jul 28 17:16:58.684843 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:58.684752 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-fbxpj" event={"ID":"2487c85c-45bb-4eea-930a-1e8493b3a1f0","Type":"ContainerDied","Data":"9af723e3b5fbf0176d07e9945f6853c8f0c6425d56c7befee835a6e81a2860f9"} Jul 28 17:16:58.684843 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:58.684806 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-fbxpj" event={"ID":"2487c85c-45bb-4eea-930a-1e8493b3a1f0","Type":"ContainerDied","Data":"a8715ade56f0cefdd8e9b1c9df34b778c17af9e13b078467c7491703ce550261"} Jul 28 17:16:58.684843 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:58.684828 2570 scope.go:117] "RemoveContainer" containerID="9af723e3b5fbf0176d07e9945f6853c8f0c6425d56c7befee835a6e81a2860f9" Jul 28 17:16:58.693349 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:58.693331 2570 scope.go:117] "RemoveContainer" containerID="9af723e3b5fbf0176d07e9945f6853c8f0c6425d56c7befee835a6e81a2860f9" Jul 28 17:16:58.693602 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:16:58.693583 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9af723e3b5fbf0176d07e9945f6853c8f0c6425d56c7befee835a6e81a2860f9\": container with ID starting with 9af723e3b5fbf0176d07e9945f6853c8f0c6425d56c7befee835a6e81a2860f9 not found: ID does not exist" containerID="9af723e3b5fbf0176d07e9945f6853c8f0c6425d56c7befee835a6e81a2860f9" Jul 28 17:16:58.693647 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:58.693612 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9af723e3b5fbf0176d07e9945f6853c8f0c6425d56c7befee835a6e81a2860f9"} err="failed to get container status \"9af723e3b5fbf0176d07e9945f6853c8f0c6425d56c7befee835a6e81a2860f9\": rpc error: code = NotFound desc = could not find container \"9af723e3b5fbf0176d07e9945f6853c8f0c6425d56c7befee835a6e81a2860f9\": container with ID starting with 9af723e3b5fbf0176d07e9945f6853c8f0c6425d56c7befee835a6e81a2860f9 not found: ID does not exist" Jul 28 17:16:58.713733 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:58.713711 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fbxpj"] Jul 28 17:16:58.715844 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:58.715822 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fbxpj"] Jul 28 17:16:58.736352 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:58.736326 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kq64z\" (UniqueName: \"kubernetes.io/projected/2487c85c-45bb-4eea-930a-1e8493b3a1f0-kube-api-access-kq64z\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:16:58.736352 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:16:58.736352 2570 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2487c85c-45bb-4eea-930a-1e8493b3a1f0-config-file\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:17:00.119818 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:00.119783 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2487c85c-45bb-4eea-930a-1e8493b3a1f0" path="/var/lib/kubelet/pods/2487c85c-45bb-4eea-930a-1e8493b3a1f0/volumes" Jul 28 17:17:00.846101 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:00.846069 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-9kxc6"] Jul 28 17:17:00.846453 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:00.846440 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2487c85c-45bb-4eea-930a-1e8493b3a1f0" containerName="limitador" Jul 28 17:17:00.846518 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:00.846454 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="2487c85c-45bb-4eea-930a-1e8493b3a1f0" containerName="limitador" Jul 28 17:17:00.846518 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:00.846477 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="447ccc2b-9fa7-4efa-9d76-65b933b6cd06" containerName="authorino" Jul 28 17:17:00.846518 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:00.846483 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="447ccc2b-9fa7-4efa-9d76-65b933b6cd06" containerName="authorino" Jul 28 17:17:00.846616 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:00.846542 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="2487c85c-45bb-4eea-930a-1e8493b3a1f0" containerName="limitador" Jul 28 17:17:00.846616 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:00.846549 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="447ccc2b-9fa7-4efa-9d76-65b933b6cd06" containerName="authorino" Jul 28 17:17:00.850810 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:00.850785 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9kxc6" Jul 28 17:17:00.855094 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:00.855070 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 28 17:17:00.855586 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:00.855568 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-dntvq\"" Jul 28 17:17:00.856606 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:00.856588 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 28 17:17:00.856729 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:00.856708 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 28 17:17:00.875412 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:00.875385 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-9kxc6"] Jul 28 17:17:00.955216 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:00.955181 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c7a628d7-7c96-4a8a-87f7-abb463d1dfa8-data\") pod \"postgres-69fb4594cf-9kxc6\" (UID: \"c7a628d7-7c96-4a8a-87f7-abb463d1dfa8\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9kxc6" Jul 28 17:17:00.955216 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:00.955215 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-256jj\" (UniqueName: \"kubernetes.io/projected/c7a628d7-7c96-4a8a-87f7-abb463d1dfa8-kube-api-access-256jj\") pod \"postgres-69fb4594cf-9kxc6\" (UID: \"c7a628d7-7c96-4a8a-87f7-abb463d1dfa8\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9kxc6" Jul 28 17:17:01.056521 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:01.056469 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c7a628d7-7c96-4a8a-87f7-abb463d1dfa8-data\") pod \"postgres-69fb4594cf-9kxc6\" (UID: \"c7a628d7-7c96-4a8a-87f7-abb463d1dfa8\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9kxc6" Jul 28 17:17:01.056521 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:01.056525 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-256jj\" (UniqueName: \"kubernetes.io/projected/c7a628d7-7c96-4a8a-87f7-abb463d1dfa8-kube-api-access-256jj\") pod \"postgres-69fb4594cf-9kxc6\" (UID: \"c7a628d7-7c96-4a8a-87f7-abb463d1dfa8\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9kxc6" Jul 28 17:17:01.056876 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:01.056857 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c7a628d7-7c96-4a8a-87f7-abb463d1dfa8-data\") pod \"postgres-69fb4594cf-9kxc6\" (UID: \"c7a628d7-7c96-4a8a-87f7-abb463d1dfa8\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9kxc6" Jul 28 17:17:01.073966 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:01.073940 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-256jj\" (UniqueName: \"kubernetes.io/projected/c7a628d7-7c96-4a8a-87f7-abb463d1dfa8-kube-api-access-256jj\") pod \"postgres-69fb4594cf-9kxc6\" (UID: \"c7a628d7-7c96-4a8a-87f7-abb463d1dfa8\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9kxc6" Jul 28 17:17:01.161190 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:01.161164 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9kxc6" Jul 28 17:17:01.317604 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:01.317578 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-9kxc6"] Jul 28 17:17:01.319678 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:17:01.319644 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7a628d7_7c96_4a8a_87f7_abb463d1dfa8.slice/crio-251562e139919c41b70375c3fca594680d35bb9d95fbbfafdef94d32f1b6ae0b WatchSource:0}: Error finding container 251562e139919c41b70375c3fca594680d35bb9d95fbbfafdef94d32f1b6ae0b: Status 404 returned error can't find the container with id 251562e139919c41b70375c3fca594680d35bb9d95fbbfafdef94d32f1b6ae0b Jul 28 17:17:01.701463 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:01.701427 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9kxc6" event={"ID":"c7a628d7-7c96-4a8a-87f7-abb463d1dfa8","Type":"ContainerStarted","Data":"251562e139919c41b70375c3fca594680d35bb9d95fbbfafdef94d32f1b6ae0b"} Jul 28 17:17:07.742168 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:07.742129 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9kxc6" event={"ID":"c7a628d7-7c96-4a8a-87f7-abb463d1dfa8","Type":"ContainerStarted","Data":"a3b0f6145f949f52bddf9afdfd2bb427de74072228d789368ab7185807244099"} Jul 28 17:17:07.742540 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:07.742197 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9kxc6" Jul 28 17:17:07.767575 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:07.767528 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9kxc6" podStartSLOduration=1.9825129289999999 podStartE2EDuration="7.76751346s" totalImagesPullingTime="5.785000531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:17:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:17:07.76645942 +0000 UTC m=+526.375390812" watchObservedRunningTime="2026-07-28 17:17:07.76751346 +0000 UTC m=+526.376444830" Jul 28 17:17:13.776432 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:13.776402 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9kxc6" Jul 28 17:17:14.449659 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:14.449577 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7vkxx"] Jul 28 17:17:14.456287 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:14.456267 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7vkxx" Jul 28 17:17:14.470235 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:14.470208 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7vkxx"] Jul 28 17:17:14.578598 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:14.578567 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvjfn\" (UniqueName: \"kubernetes.io/projected/8550743a-dee6-4996-85fe-a97eae917c7c-kube-api-access-qvjfn\") pod \"authorino-5d4b67c446-7vkxx\" (UID: \"8550743a-dee6-4996-85fe-a97eae917c7c\") " pod="kuadrant-system/authorino-5d4b67c446-7vkxx" Jul 28 17:17:14.679977 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:14.679937 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvjfn\" (UniqueName: \"kubernetes.io/projected/8550743a-dee6-4996-85fe-a97eae917c7c-kube-api-access-qvjfn\") pod \"authorino-5d4b67c446-7vkxx\" (UID: \"8550743a-dee6-4996-85fe-a97eae917c7c\") " pod="kuadrant-system/authorino-5d4b67c446-7vkxx" Jul 28 17:17:14.691001 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:14.690974 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvjfn\" (UniqueName: \"kubernetes.io/projected/8550743a-dee6-4996-85fe-a97eae917c7c-kube-api-access-qvjfn\") pod \"authorino-5d4b67c446-7vkxx\" (UID: \"8550743a-dee6-4996-85fe-a97eae917c7c\") " pod="kuadrant-system/authorino-5d4b67c446-7vkxx" Jul 28 17:17:14.710716 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:14.710659 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7vkxx"] Jul 28 17:17:14.710975 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:14.710960 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7vkxx" Jul 28 17:17:14.762441 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:14.762407 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5c96bd887-tzpcz"] Jul 28 17:17:14.768702 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:14.768629 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5c96bd887-tzpcz" Jul 28 17:17:14.783286 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:14.782831 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5c96bd887-tzpcz"] Jul 28 17:17:14.884752 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:14.884718 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d42z5\" (UniqueName: \"kubernetes.io/projected/5c63af7e-711f-4e94-8d1d-8d74e21a9f95-kube-api-access-d42z5\") pod \"authorino-5c96bd887-tzpcz\" (UID: \"5c63af7e-711f-4e94-8d1d-8d74e21a9f95\") " pod="kuadrant-system/authorino-5c96bd887-tzpcz" Jul 28 17:17:14.889274 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:14.889252 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7vkxx"] Jul 28 17:17:14.890480 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:17:14.890452 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8550743a_dee6_4996_85fe_a97eae917c7c.slice/crio-91efb90c3606ec3193820a55d835990cf8ebfa4ac14b0de4fe3ae13586c251f5 WatchSource:0}: Error finding container 91efb90c3606ec3193820a55d835990cf8ebfa4ac14b0de4fe3ae13586c251f5: Status 404 returned error can't find the container with id 91efb90c3606ec3193820a55d835990cf8ebfa4ac14b0de4fe3ae13586c251f5 Jul 28 17:17:14.927422 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:14.927392 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5c96bd887-tzpcz"] Jul 28 17:17:14.927713 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:17:14.927694 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-d42z5], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5c96bd887-tzpcz" podUID="5c63af7e-711f-4e94-8d1d-8d74e21a9f95" Jul 28 17:17:14.985926 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:14.985838 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d42z5\" (UniqueName: \"kubernetes.io/projected/5c63af7e-711f-4e94-8d1d-8d74e21a9f95-kube-api-access-d42z5\") pod \"authorino-5c96bd887-tzpcz\" (UID: \"5c63af7e-711f-4e94-8d1d-8d74e21a9f95\") " pod="kuadrant-system/authorino-5c96bd887-tzpcz" Jul 28 17:17:14.988466 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:14.988441 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-d8df5c7b7-c28tl"] Jul 28 17:17:14.992770 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:14.992749 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d8df5c7b7-c28tl" Jul 28 17:17:14.996483 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:14.996455 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 28 17:17:15.001915 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:15.001897 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d42z5\" (UniqueName: \"kubernetes.io/projected/5c63af7e-711f-4e94-8d1d-8d74e21a9f95-kube-api-access-d42z5\") pod \"authorino-5c96bd887-tzpcz\" (UID: \"5c63af7e-711f-4e94-8d1d-8d74e21a9f95\") " pod="kuadrant-system/authorino-5c96bd887-tzpcz" Jul 28 17:17:15.010497 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:15.010477 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-d8df5c7b7-c28tl"] Jul 28 17:17:15.086975 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:15.086944 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/028cacb5-b9d3-402f-8a34-56621ebded34-tls-cert\") pod \"authorino-d8df5c7b7-c28tl\" (UID: \"028cacb5-b9d3-402f-8a34-56621ebded34\") " pod="kuadrant-system/authorino-d8df5c7b7-c28tl" Jul 28 17:17:15.087163 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:15.087023 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sf4v9\" (UniqueName: \"kubernetes.io/projected/028cacb5-b9d3-402f-8a34-56621ebded34-kube-api-access-sf4v9\") pod \"authorino-d8df5c7b7-c28tl\" (UID: \"028cacb5-b9d3-402f-8a34-56621ebded34\") " pod="kuadrant-system/authorino-d8df5c7b7-c28tl" Jul 28 17:17:15.187653 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:15.187619 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sf4v9\" (UniqueName: \"kubernetes.io/projected/028cacb5-b9d3-402f-8a34-56621ebded34-kube-api-access-sf4v9\") pod \"authorino-d8df5c7b7-c28tl\" (UID: \"028cacb5-b9d3-402f-8a34-56621ebded34\") " pod="kuadrant-system/authorino-d8df5c7b7-c28tl" Jul 28 17:17:15.187838 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:15.187671 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/028cacb5-b9d3-402f-8a34-56621ebded34-tls-cert\") pod \"authorino-d8df5c7b7-c28tl\" (UID: \"028cacb5-b9d3-402f-8a34-56621ebded34\") " pod="kuadrant-system/authorino-d8df5c7b7-c28tl" Jul 28 17:17:15.190078 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:15.190057 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/028cacb5-b9d3-402f-8a34-56621ebded34-tls-cert\") pod \"authorino-d8df5c7b7-c28tl\" (UID: \"028cacb5-b9d3-402f-8a34-56621ebded34\") " pod="kuadrant-system/authorino-d8df5c7b7-c28tl" Jul 28 17:17:15.203289 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:15.203262 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sf4v9\" (UniqueName: \"kubernetes.io/projected/028cacb5-b9d3-402f-8a34-56621ebded34-kube-api-access-sf4v9\") pod \"authorino-d8df5c7b7-c28tl\" (UID: \"028cacb5-b9d3-402f-8a34-56621ebded34\") " pod="kuadrant-system/authorino-d8df5c7b7-c28tl" Jul 28 17:17:15.303453 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:15.303359 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d8df5c7b7-c28tl" Jul 28 17:17:15.445142 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:15.445094 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-d8df5c7b7-c28tl"] Jul 28 17:17:15.446136 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:17:15.446082 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod028cacb5_b9d3_402f_8a34_56621ebded34.slice/crio-6d441b1644f5a8683ff2d20ed05e6e0f60249901d2fbbcda76d8b8722de2541a WatchSource:0}: Error finding container 6d441b1644f5a8683ff2d20ed05e6e0f60249901d2fbbcda76d8b8722de2541a: Status 404 returned error can't find the container with id 6d441b1644f5a8683ff2d20ed05e6e0f60249901d2fbbcda76d8b8722de2541a Jul 28 17:17:15.794025 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:15.793987 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d8df5c7b7-c28tl" event={"ID":"028cacb5-b9d3-402f-8a34-56621ebded34","Type":"ContainerStarted","Data":"6d441b1644f5a8683ff2d20ed05e6e0f60249901d2fbbcda76d8b8722de2541a"} Jul 28 17:17:15.795426 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:15.795400 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5c96bd887-tzpcz" Jul 28 17:17:15.795529 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:15.795420 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-7vkxx" event={"ID":"8550743a-dee6-4996-85fe-a97eae917c7c","Type":"ContainerStarted","Data":"bf8f35e5501536938678a66ada4c74cdbc52a0770d8559e55e4c63ee505dc7a0"} Jul 28 17:17:15.795529 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:15.795459 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-7vkxx" event={"ID":"8550743a-dee6-4996-85fe-a97eae917c7c","Type":"ContainerStarted","Data":"91efb90c3606ec3193820a55d835990cf8ebfa4ac14b0de4fe3ae13586c251f5"} Jul 28 17:17:15.795529 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:15.795463 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-7vkxx" podUID="8550743a-dee6-4996-85fe-a97eae917c7c" containerName="authorino" containerID="cri-o://bf8f35e5501536938678a66ada4c74cdbc52a0770d8559e55e4c63ee505dc7a0" gracePeriod=30 Jul 28 17:17:15.802852 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:15.802834 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5c96bd887-tzpcz" Jul 28 17:17:15.851021 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:15.850970 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-7vkxx" podStartSLOduration=1.2761177830000001 podStartE2EDuration="1.850954444s" totalImagesPullingTime="574.836661ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:17:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:17:15.820772738 +0000 UTC m=+534.429704135" watchObservedRunningTime="2026-07-28 17:17:15.850954444 +0000 UTC m=+534.459885813" Jul 28 17:17:15.993805 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:15.993776 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d42z5\" (UniqueName: \"kubernetes.io/projected/5c63af7e-711f-4e94-8d1d-8d74e21a9f95-kube-api-access-d42z5\") pod \"5c63af7e-711f-4e94-8d1d-8d74e21a9f95\" (UID: \"5c63af7e-711f-4e94-8d1d-8d74e21a9f95\") " Jul 28 17:17:15.995655 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:15.995627 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c63af7e-711f-4e94-8d1d-8d74e21a9f95-kube-api-access-d42z5" (OuterVolumeSpecName: "kube-api-access-d42z5") pod "5c63af7e-711f-4e94-8d1d-8d74e21a9f95" (UID: "5c63af7e-711f-4e94-8d1d-8d74e21a9f95"). InnerVolumeSpecName "kube-api-access-d42z5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:17:16.040926 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:16.040901 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7vkxx" Jul 28 17:17:16.095357 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:16.095215 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d42z5\" (UniqueName: \"kubernetes.io/projected/5c63af7e-711f-4e94-8d1d-8d74e21a9f95-kube-api-access-d42z5\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:17:16.196155 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:16.196095 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qvjfn\" (UniqueName: \"kubernetes.io/projected/8550743a-dee6-4996-85fe-a97eae917c7c-kube-api-access-qvjfn\") pod \"8550743a-dee6-4996-85fe-a97eae917c7c\" (UID: \"8550743a-dee6-4996-85fe-a97eae917c7c\") " Jul 28 17:17:16.198140 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:16.198099 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8550743a-dee6-4996-85fe-a97eae917c7c-kube-api-access-qvjfn" (OuterVolumeSpecName: "kube-api-access-qvjfn") pod "8550743a-dee6-4996-85fe-a97eae917c7c" (UID: "8550743a-dee6-4996-85fe-a97eae917c7c"). InnerVolumeSpecName "kube-api-access-qvjfn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:17:16.296967 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:16.296934 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvjfn\" (UniqueName: \"kubernetes.io/projected/8550743a-dee6-4996-85fe-a97eae917c7c-kube-api-access-qvjfn\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:17:16.802206 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:16.802170 2570 generic.go:356] "Generic (PLEG): container finished" podID="8550743a-dee6-4996-85fe-a97eae917c7c" containerID="bf8f35e5501536938678a66ada4c74cdbc52a0770d8559e55e4c63ee505dc7a0" exitCode=0 Jul 28 17:17:16.802648 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:16.802216 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7vkxx" Jul 28 17:17:16.802648 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:16.802255 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-7vkxx" event={"ID":"8550743a-dee6-4996-85fe-a97eae917c7c","Type":"ContainerDied","Data":"bf8f35e5501536938678a66ada4c74cdbc52a0770d8559e55e4c63ee505dc7a0"} Jul 28 17:17:16.802648 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:16.802291 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-7vkxx" event={"ID":"8550743a-dee6-4996-85fe-a97eae917c7c","Type":"ContainerDied","Data":"91efb90c3606ec3193820a55d835990cf8ebfa4ac14b0de4fe3ae13586c251f5"} Jul 28 17:17:16.802648 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:16.802312 2570 scope.go:117] "RemoveContainer" containerID="bf8f35e5501536938678a66ada4c74cdbc52a0770d8559e55e4c63ee505dc7a0" Jul 28 17:17:16.803905 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:16.803885 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5c96bd887-tzpcz" Jul 28 17:17:16.804034 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:16.803887 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d8df5c7b7-c28tl" event={"ID":"028cacb5-b9d3-402f-8a34-56621ebded34","Type":"ContainerStarted","Data":"088977583abe04045bf81b6956de34a1a2f0482d4e377273679a296344b1a2a4"} Jul 28 17:17:16.811098 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:16.811079 2570 scope.go:117] "RemoveContainer" containerID="bf8f35e5501536938678a66ada4c74cdbc52a0770d8559e55e4c63ee505dc7a0" Jul 28 17:17:16.811383 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:17:16.811365 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf8f35e5501536938678a66ada4c74cdbc52a0770d8559e55e4c63ee505dc7a0\": container with ID starting with bf8f35e5501536938678a66ada4c74cdbc52a0770d8559e55e4c63ee505dc7a0 not found: ID does not exist" containerID="bf8f35e5501536938678a66ada4c74cdbc52a0770d8559e55e4c63ee505dc7a0" Jul 28 17:17:16.811435 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:16.811393 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf8f35e5501536938678a66ada4c74cdbc52a0770d8559e55e4c63ee505dc7a0"} err="failed to get container status \"bf8f35e5501536938678a66ada4c74cdbc52a0770d8559e55e4c63ee505dc7a0\": rpc error: code = NotFound desc = could not find container \"bf8f35e5501536938678a66ada4c74cdbc52a0770d8559e55e4c63ee505dc7a0\": container with ID starting with bf8f35e5501536938678a66ada4c74cdbc52a0770d8559e55e4c63ee505dc7a0 not found: ID does not exist" Jul 28 17:17:16.845524 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:16.845491 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5c96bd887-tzpcz"] Jul 28 17:17:16.847348 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:16.847324 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5c96bd887-tzpcz"] Jul 28 17:17:16.898631 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:16.898554 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-d8df5c7b7-c28tl" podStartSLOduration=2.409042481 podStartE2EDuration="2.898529557s" totalImagesPullingTime="489.487076ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:17:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:17:16.870765157 +0000 UTC m=+535.479696538" watchObservedRunningTime="2026-07-28 17:17:16.898529557 +0000 UTC m=+535.507460927" Jul 28 17:17:16.901565 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:16.901541 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7vkxx"] Jul 28 17:17:16.904582 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:16.904553 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7vkxx"] Jul 28 17:17:16.938619 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:16.938591 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-m5tpb"] Jul 28 17:17:16.938773 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:16.938752 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-m5tpb" podUID="c57fe779-a6fd-4394-9406-7de3cafb39e5" containerName="authorino" containerID="cri-o://b1b60341fac93356fff9b6f4561b83471bfefc8483ee9dfc514c8d37254c1d16" gracePeriod=30 Jul 28 17:17:17.199933 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:17.199907 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-m5tpb" Jul 28 17:17:17.306232 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:17.306151 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mfvnl\" (UniqueName: \"kubernetes.io/projected/c57fe779-a6fd-4394-9406-7de3cafb39e5-kube-api-access-mfvnl\") pod \"c57fe779-a6fd-4394-9406-7de3cafb39e5\" (UID: \"c57fe779-a6fd-4394-9406-7de3cafb39e5\") " Jul 28 17:17:17.308228 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:17.308200 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c57fe779-a6fd-4394-9406-7de3cafb39e5-kube-api-access-mfvnl" (OuterVolumeSpecName: "kube-api-access-mfvnl") pod "c57fe779-a6fd-4394-9406-7de3cafb39e5" (UID: "c57fe779-a6fd-4394-9406-7de3cafb39e5"). InnerVolumeSpecName "kube-api-access-mfvnl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:17:17.407091 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:17.407059 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfvnl\" (UniqueName: \"kubernetes.io/projected/c57fe779-a6fd-4394-9406-7de3cafb39e5-kube-api-access-mfvnl\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:17:17.810544 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:17.810513 2570 generic.go:356] "Generic (PLEG): container finished" podID="c57fe779-a6fd-4394-9406-7de3cafb39e5" containerID="b1b60341fac93356fff9b6f4561b83471bfefc8483ee9dfc514c8d37254c1d16" exitCode=0 Jul 28 17:17:17.810925 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:17.810555 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-m5tpb" Jul 28 17:17:17.810925 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:17.810603 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-m5tpb" event={"ID":"c57fe779-a6fd-4394-9406-7de3cafb39e5","Type":"ContainerDied","Data":"b1b60341fac93356fff9b6f4561b83471bfefc8483ee9dfc514c8d37254c1d16"} Jul 28 17:17:17.810925 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:17.810644 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-m5tpb" event={"ID":"c57fe779-a6fd-4394-9406-7de3cafb39e5","Type":"ContainerDied","Data":"7ad7223613ccde65a265f351383f3b69e93451619c07d7e9221bf9c9790c7cfc"} Jul 28 17:17:17.810925 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:17.810693 2570 scope.go:117] "RemoveContainer" containerID="b1b60341fac93356fff9b6f4561b83471bfefc8483ee9dfc514c8d37254c1d16" Jul 28 17:17:17.820654 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:17.820637 2570 scope.go:117] "RemoveContainer" containerID="b1b60341fac93356fff9b6f4561b83471bfefc8483ee9dfc514c8d37254c1d16" Jul 28 17:17:17.820905 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:17:17.820885 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1b60341fac93356fff9b6f4561b83471bfefc8483ee9dfc514c8d37254c1d16\": container with ID starting with b1b60341fac93356fff9b6f4561b83471bfefc8483ee9dfc514c8d37254c1d16 not found: ID does not exist" containerID="b1b60341fac93356fff9b6f4561b83471bfefc8483ee9dfc514c8d37254c1d16" Jul 28 17:17:17.820954 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:17.820912 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1b60341fac93356fff9b6f4561b83471bfefc8483ee9dfc514c8d37254c1d16"} err="failed to get container status \"b1b60341fac93356fff9b6f4561b83471bfefc8483ee9dfc514c8d37254c1d16\": rpc error: code = NotFound desc = could not find container \"b1b60341fac93356fff9b6f4561b83471bfefc8483ee9dfc514c8d37254c1d16\": container with ID starting with b1b60341fac93356fff9b6f4561b83471bfefc8483ee9dfc514c8d37254c1d16 not found: ID does not exist" Jul 28 17:17:17.881424 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:17.881399 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-m5tpb"] Jul 28 17:17:17.893405 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:17.893379 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-m5tpb"] Jul 28 17:17:18.119497 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:18.119471 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c63af7e-711f-4e94-8d1d-8d74e21a9f95" path="/var/lib/kubelet/pods/5c63af7e-711f-4e94-8d1d-8d74e21a9f95/volumes" Jul 28 17:17:18.119711 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:18.119699 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8550743a-dee6-4996-85fe-a97eae917c7c" path="/var/lib/kubelet/pods/8550743a-dee6-4996-85fe-a97eae917c7c/volumes" Jul 28 17:17:18.120002 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:18.119990 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c57fe779-a6fd-4394-9406-7de3cafb39e5" path="/var/lib/kubelet/pods/c57fe779-a6fd-4394-9406-7de3cafb39e5/volumes" Jul 28 17:17:22.711175 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:22.711138 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-686cb44cc4-t2ltn"] Jul 28 17:17:22.711545 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:22.711532 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c57fe779-a6fd-4394-9406-7de3cafb39e5" containerName="authorino" Jul 28 17:17:22.711545 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:22.711543 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="c57fe779-a6fd-4394-9406-7de3cafb39e5" containerName="authorino" Jul 28 17:17:22.711616 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:22.711553 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8550743a-dee6-4996-85fe-a97eae917c7c" containerName="authorino" Jul 28 17:17:22.711616 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:22.711559 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="8550743a-dee6-4996-85fe-a97eae917c7c" containerName="authorino" Jul 28 17:17:22.711676 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:22.711619 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="c57fe779-a6fd-4394-9406-7de3cafb39e5" containerName="authorino" Jul 28 17:17:22.711676 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:22.711627 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="8550743a-dee6-4996-85fe-a97eae917c7c" containerName="authorino" Jul 28 17:17:22.728830 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:22.728798 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-686cb44cc4-t2ltn" Jul 28 17:17:22.731712 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:22.731687 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-z2rlk\"" Jul 28 17:17:22.732143 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:22.732105 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 28 17:17:22.732247 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:22.732235 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 28 17:17:22.741706 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:22.741681 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-686cb44cc4-t2ltn"] Jul 28 17:17:22.750083 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:22.750059 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kqfr\" (UniqueName: \"kubernetes.io/projected/d7d70a8b-3fcd-4bbc-a52b-1585e791e418-kube-api-access-6kqfr\") pod \"maas-controller-686cb44cc4-t2ltn\" (UID: \"d7d70a8b-3fcd-4bbc-a52b-1585e791e418\") " pod="opendatahub/maas-controller-686cb44cc4-t2ltn" Jul 28 17:17:22.750226 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:22.750196 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d7d70a8b-3fcd-4bbc-a52b-1585e791e418-webhook-cert\") pod \"maas-controller-686cb44cc4-t2ltn\" (UID: \"d7d70a8b-3fcd-4bbc-a52b-1585e791e418\") " pod="opendatahub/maas-controller-686cb44cc4-t2ltn" Jul 28 17:17:22.851236 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:22.851202 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d7d70a8b-3fcd-4bbc-a52b-1585e791e418-webhook-cert\") pod \"maas-controller-686cb44cc4-t2ltn\" (UID: \"d7d70a8b-3fcd-4bbc-a52b-1585e791e418\") " pod="opendatahub/maas-controller-686cb44cc4-t2ltn" Jul 28 17:17:22.851398 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:22.851256 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kqfr\" (UniqueName: \"kubernetes.io/projected/d7d70a8b-3fcd-4bbc-a52b-1585e791e418-kube-api-access-6kqfr\") pod \"maas-controller-686cb44cc4-t2ltn\" (UID: \"d7d70a8b-3fcd-4bbc-a52b-1585e791e418\") " pod="opendatahub/maas-controller-686cb44cc4-t2ltn" Jul 28 17:17:22.853632 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:22.853604 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d7d70a8b-3fcd-4bbc-a52b-1585e791e418-webhook-cert\") pod \"maas-controller-686cb44cc4-t2ltn\" (UID: \"d7d70a8b-3fcd-4bbc-a52b-1585e791e418\") " pod="opendatahub/maas-controller-686cb44cc4-t2ltn" Jul 28 17:17:22.865229 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:22.865200 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kqfr\" (UniqueName: \"kubernetes.io/projected/d7d70a8b-3fcd-4bbc-a52b-1585e791e418-kube-api-access-6kqfr\") pod \"maas-controller-686cb44cc4-t2ltn\" (UID: \"d7d70a8b-3fcd-4bbc-a52b-1585e791e418\") " pod="opendatahub/maas-controller-686cb44cc4-t2ltn" Jul 28 17:17:22.962903 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:22.962822 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7d6f646b66-vlsbm"] Jul 28 17:17:22.966513 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:22.966498 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d6f646b66-vlsbm" Jul 28 17:17:22.980315 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:22.980289 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d6f646b66-vlsbm"] Jul 28 17:17:23.039135 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:23.039084 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-686cb44cc4-t2ltn" Jul 28 17:17:23.053535 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:23.053509 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9d25\" (UniqueName: \"kubernetes.io/projected/517748fc-2967-4965-8247-a82c806cee67-kube-api-access-d9d25\") pod \"maas-controller-7d6f646b66-vlsbm\" (UID: \"517748fc-2967-4965-8247-a82c806cee67\") " pod="opendatahub/maas-controller-7d6f646b66-vlsbm" Jul 28 17:17:23.053648 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:23.053632 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/517748fc-2967-4965-8247-a82c806cee67-webhook-cert\") pod \"maas-controller-7d6f646b66-vlsbm\" (UID: \"517748fc-2967-4965-8247-a82c806cee67\") " pod="opendatahub/maas-controller-7d6f646b66-vlsbm" Jul 28 17:17:23.154829 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:23.154795 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/517748fc-2967-4965-8247-a82c806cee67-webhook-cert\") pod \"maas-controller-7d6f646b66-vlsbm\" (UID: \"517748fc-2967-4965-8247-a82c806cee67\") " pod="opendatahub/maas-controller-7d6f646b66-vlsbm" Jul 28 17:17:23.154968 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:23.154856 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9d25\" (UniqueName: \"kubernetes.io/projected/517748fc-2967-4965-8247-a82c806cee67-kube-api-access-d9d25\") pod \"maas-controller-7d6f646b66-vlsbm\" (UID: \"517748fc-2967-4965-8247-a82c806cee67\") " pod="opendatahub/maas-controller-7d6f646b66-vlsbm" Jul 28 17:17:23.157268 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:23.157244 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/517748fc-2967-4965-8247-a82c806cee67-webhook-cert\") pod \"maas-controller-7d6f646b66-vlsbm\" (UID: \"517748fc-2967-4965-8247-a82c806cee67\") " pod="opendatahub/maas-controller-7d6f646b66-vlsbm" Jul 28 17:17:23.173546 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:23.173526 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9d25\" (UniqueName: \"kubernetes.io/projected/517748fc-2967-4965-8247-a82c806cee67-kube-api-access-d9d25\") pod \"maas-controller-7d6f646b66-vlsbm\" (UID: \"517748fc-2967-4965-8247-a82c806cee67\") " pod="opendatahub/maas-controller-7d6f646b66-vlsbm" Jul 28 17:17:23.204346 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:23.204325 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-686cb44cc4-t2ltn"] Jul 28 17:17:23.206310 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:17:23.206280 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7d70a8b_3fcd_4bbc_a52b_1585e791e418.slice/crio-1e4f7d5188c2bc32505b0ddc19bbe73270158907b1747d18e4be7ffc5f0e4807 WatchSource:0}: Error finding container 1e4f7d5188c2bc32505b0ddc19bbe73270158907b1747d18e4be7ffc5f0e4807: Status 404 returned error can't find the container with id 1e4f7d5188c2bc32505b0ddc19bbe73270158907b1747d18e4be7ffc5f0e4807 Jul 28 17:17:23.276858 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:23.276774 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d6f646b66-vlsbm" Jul 28 17:17:23.420470 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:23.420438 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d6f646b66-vlsbm"] Jul 28 17:17:23.423442 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:17:23.423418 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod517748fc_2967_4965_8247_a82c806cee67.slice/crio-a511d0610204db5ac69a27dd80afc27f3864ab69a9fba6241b589c9879ab2677 WatchSource:0}: Error finding container a511d0610204db5ac69a27dd80afc27f3864ab69a9fba6241b589c9879ab2677: Status 404 returned error can't find the container with id a511d0610204db5ac69a27dd80afc27f3864ab69a9fba6241b589c9879ab2677 Jul 28 17:17:23.843488 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:23.843447 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d6f646b66-vlsbm" event={"ID":"517748fc-2967-4965-8247-a82c806cee67","Type":"ContainerStarted","Data":"a511d0610204db5ac69a27dd80afc27f3864ab69a9fba6241b589c9879ab2677"} Jul 28 17:17:23.844539 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:23.844514 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-686cb44cc4-t2ltn" event={"ID":"d7d70a8b-3fcd-4bbc-a52b-1585e791e418","Type":"ContainerStarted","Data":"1e4f7d5188c2bc32505b0ddc19bbe73270158907b1747d18e4be7ffc5f0e4807"} Jul 28 17:17:26.867947 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:26.867910 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d6f646b66-vlsbm" event={"ID":"517748fc-2967-4965-8247-a82c806cee67","Type":"ContainerStarted","Data":"1f1ddab83675ff70484710921498be73d43e1bebea3ee5c170a9a22482b190f3"} Jul 28 17:17:26.868431 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:26.868026 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7d6f646b66-vlsbm" Jul 28 17:17:26.869480 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:26.869452 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-686cb44cc4-t2ltn" event={"ID":"d7d70a8b-3fcd-4bbc-a52b-1585e791e418","Type":"ContainerStarted","Data":"c4c1ce5a860ea02a9c057dc05e17870de10beb9ec477b858299e5679bd634add"} Jul 28 17:17:26.869605 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:26.869590 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-686cb44cc4-t2ltn" Jul 28 17:17:26.901441 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:26.901405 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7d6f646b66-vlsbm" podStartSLOduration=2.182055786 podStartE2EDuration="4.901392293s" totalImagesPullingTime="2.719336507s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:17:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:17:26.901047217 +0000 UTC m=+545.509978591" watchObservedRunningTime="2026-07-28 17:17:26.901392293 +0000 UTC m=+545.510323664" Jul 28 17:17:26.933979 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:26.933938 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-686cb44cc4-t2ltn" podStartSLOduration=1.990425424 podStartE2EDuration="4.933923988s" totalImagesPullingTime="2.943498564s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:17:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:17:26.933204966 +0000 UTC m=+545.542136351" watchObservedRunningTime="2026-07-28 17:17:26.933923988 +0000 UTC m=+545.542855357" Jul 28 17:17:37.884095 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:37.884062 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7d6f646b66-vlsbm" Jul 28 17:17:37.884491 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:37.884243 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-686cb44cc4-t2ltn" Jul 28 17:17:37.998337 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:37.998304 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-686cb44cc4-t2ltn"] Jul 28 17:17:37.998516 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:37.998494 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-686cb44cc4-t2ltn" podUID="d7d70a8b-3fcd-4bbc-a52b-1585e791e418" containerName="manager" containerID="cri-o://c4c1ce5a860ea02a9c057dc05e17870de10beb9ec477b858299e5679bd634add" gracePeriod=30 Jul 28 17:17:38.251260 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:38.251235 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-686cb44cc4-t2ltn" Jul 28 17:17:38.293634 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:38.293603 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d7d70a8b-3fcd-4bbc-a52b-1585e791e418-webhook-cert\") pod \"d7d70a8b-3fcd-4bbc-a52b-1585e791e418\" (UID: \"d7d70a8b-3fcd-4bbc-a52b-1585e791e418\") " Jul 28 17:17:38.293814 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:38.293707 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6kqfr\" (UniqueName: \"kubernetes.io/projected/d7d70a8b-3fcd-4bbc-a52b-1585e791e418-kube-api-access-6kqfr\") pod \"d7d70a8b-3fcd-4bbc-a52b-1585e791e418\" (UID: \"d7d70a8b-3fcd-4bbc-a52b-1585e791e418\") " Jul 28 17:17:38.295789 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:38.295751 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7d70a8b-3fcd-4bbc-a52b-1585e791e418-kube-api-access-6kqfr" (OuterVolumeSpecName: "kube-api-access-6kqfr") pod "d7d70a8b-3fcd-4bbc-a52b-1585e791e418" (UID: "d7d70a8b-3fcd-4bbc-a52b-1585e791e418"). InnerVolumeSpecName "kube-api-access-6kqfr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:17:38.295898 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:38.295763 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7d70a8b-3fcd-4bbc-a52b-1585e791e418-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d7d70a8b-3fcd-4bbc-a52b-1585e791e418" (UID: "d7d70a8b-3fcd-4bbc-a52b-1585e791e418"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:17:38.395419 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:38.395379 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6kqfr\" (UniqueName: \"kubernetes.io/projected/d7d70a8b-3fcd-4bbc-a52b-1585e791e418-kube-api-access-6kqfr\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:17:38.395419 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:38.395412 2570 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d7d70a8b-3fcd-4bbc-a52b-1585e791e418-webhook-cert\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:17:38.936502 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:38.936459 2570 generic.go:356] "Generic (PLEG): container finished" podID="d7d70a8b-3fcd-4bbc-a52b-1585e791e418" containerID="c4c1ce5a860ea02a9c057dc05e17870de10beb9ec477b858299e5679bd634add" exitCode=0 Jul 28 17:17:38.936975 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:38.936518 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-686cb44cc4-t2ltn" Jul 28 17:17:38.936975 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:38.936545 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-686cb44cc4-t2ltn" event={"ID":"d7d70a8b-3fcd-4bbc-a52b-1585e791e418","Type":"ContainerDied","Data":"c4c1ce5a860ea02a9c057dc05e17870de10beb9ec477b858299e5679bd634add"} Jul 28 17:17:38.936975 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:38.936641 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-686cb44cc4-t2ltn" event={"ID":"d7d70a8b-3fcd-4bbc-a52b-1585e791e418","Type":"ContainerDied","Data":"1e4f7d5188c2bc32505b0ddc19bbe73270158907b1747d18e4be7ffc5f0e4807"} Jul 28 17:17:38.936975 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:38.936667 2570 scope.go:117] "RemoveContainer" containerID="c4c1ce5a860ea02a9c057dc05e17870de10beb9ec477b858299e5679bd634add" Jul 28 17:17:38.945858 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:38.945838 2570 scope.go:117] "RemoveContainer" containerID="c4c1ce5a860ea02a9c057dc05e17870de10beb9ec477b858299e5679bd634add" Jul 28 17:17:38.946127 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:17:38.946097 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4c1ce5a860ea02a9c057dc05e17870de10beb9ec477b858299e5679bd634add\": container with ID starting with c4c1ce5a860ea02a9c057dc05e17870de10beb9ec477b858299e5679bd634add not found: ID does not exist" containerID="c4c1ce5a860ea02a9c057dc05e17870de10beb9ec477b858299e5679bd634add" Jul 28 17:17:38.946179 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:38.946139 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4c1ce5a860ea02a9c057dc05e17870de10beb9ec477b858299e5679bd634add"} err="failed to get container status \"c4c1ce5a860ea02a9c057dc05e17870de10beb9ec477b858299e5679bd634add\": rpc error: code = NotFound desc = could not find container \"c4c1ce5a860ea02a9c057dc05e17870de10beb9ec477b858299e5679bd634add\": container with ID starting with c4c1ce5a860ea02a9c057dc05e17870de10beb9ec477b858299e5679bd634add not found: ID does not exist" Jul 28 17:17:38.962891 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:38.962867 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-686cb44cc4-t2ltn"] Jul 28 17:17:38.966881 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:38.966863 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-686cb44cc4-t2ltn"] Jul 28 17:17:40.119991 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:40.119949 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7d70a8b-3fcd-4bbc-a52b-1585e791e418" path="/var/lib/kubelet/pods/d7d70a8b-3fcd-4bbc-a52b-1585e791e418/volumes" Jul 28 17:17:56.832355 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:56.832310 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-76576b46b5-n4hkm"] Jul 28 17:17:56.832975 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:56.832954 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7d70a8b-3fcd-4bbc-a52b-1585e791e418" containerName="manager" Jul 28 17:17:56.833058 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:56.832978 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7d70a8b-3fcd-4bbc-a52b-1585e791e418" containerName="manager" Jul 28 17:17:56.833133 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:56.833076 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7d70a8b-3fcd-4bbc-a52b-1585e791e418" containerName="manager" Jul 28 17:17:56.836140 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:56.836106 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-76576b46b5-n4hkm" Jul 28 17:17:56.853268 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:56.853240 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 28 17:17:56.901788 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:56.901749 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-76576b46b5-n4hkm"] Jul 28 17:17:56.954570 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:56.954535 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thtf9\" (UniqueName: \"kubernetes.io/projected/e11a57c0-9013-4492-9722-fd0aa4bb2c92-kube-api-access-thtf9\") pod \"maas-api-76576b46b5-n4hkm\" (UID: \"e11a57c0-9013-4492-9722-fd0aa4bb2c92\") " pod="odh-ai-gateway-infra/maas-api-76576b46b5-n4hkm" Jul 28 17:17:56.954736 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:56.954577 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e11a57c0-9013-4492-9722-fd0aa4bb2c92-maas-api-tls\") pod \"maas-api-76576b46b5-n4hkm\" (UID: \"e11a57c0-9013-4492-9722-fd0aa4bb2c92\") " pod="odh-ai-gateway-infra/maas-api-76576b46b5-n4hkm" Jul 28 17:17:56.954736 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:56.954605 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e11a57c0-9013-4492-9722-fd0aa4bb2c92-openshift-service-ca\") pod \"maas-api-76576b46b5-n4hkm\" (UID: \"e11a57c0-9013-4492-9722-fd0aa4bb2c92\") " pod="odh-ai-gateway-infra/maas-api-76576b46b5-n4hkm" Jul 28 17:17:57.055774 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:57.055739 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thtf9\" (UniqueName: \"kubernetes.io/projected/e11a57c0-9013-4492-9722-fd0aa4bb2c92-kube-api-access-thtf9\") pod \"maas-api-76576b46b5-n4hkm\" (UID: \"e11a57c0-9013-4492-9722-fd0aa4bb2c92\") " pod="odh-ai-gateway-infra/maas-api-76576b46b5-n4hkm" Jul 28 17:17:57.055951 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:57.055783 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e11a57c0-9013-4492-9722-fd0aa4bb2c92-maas-api-tls\") pod \"maas-api-76576b46b5-n4hkm\" (UID: \"e11a57c0-9013-4492-9722-fd0aa4bb2c92\") " pod="odh-ai-gateway-infra/maas-api-76576b46b5-n4hkm" Jul 28 17:17:57.055951 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:57.055805 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e11a57c0-9013-4492-9722-fd0aa4bb2c92-openshift-service-ca\") pod \"maas-api-76576b46b5-n4hkm\" (UID: \"e11a57c0-9013-4492-9722-fd0aa4bb2c92\") " pod="odh-ai-gateway-infra/maas-api-76576b46b5-n4hkm" Jul 28 17:17:57.056465 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:57.056443 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e11a57c0-9013-4492-9722-fd0aa4bb2c92-openshift-service-ca\") pod \"maas-api-76576b46b5-n4hkm\" (UID: \"e11a57c0-9013-4492-9722-fd0aa4bb2c92\") " pod="odh-ai-gateway-infra/maas-api-76576b46b5-n4hkm" Jul 28 17:17:57.058350 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:57.058326 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e11a57c0-9013-4492-9722-fd0aa4bb2c92-maas-api-tls\") pod \"maas-api-76576b46b5-n4hkm\" (UID: \"e11a57c0-9013-4492-9722-fd0aa4bb2c92\") " pod="odh-ai-gateway-infra/maas-api-76576b46b5-n4hkm" Jul 28 17:17:57.079073 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:57.079041 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thtf9\" (UniqueName: \"kubernetes.io/projected/e11a57c0-9013-4492-9722-fd0aa4bb2c92-kube-api-access-thtf9\") pod \"maas-api-76576b46b5-n4hkm\" (UID: \"e11a57c0-9013-4492-9722-fd0aa4bb2c92\") " pod="odh-ai-gateway-infra/maas-api-76576b46b5-n4hkm" Jul 28 17:17:57.146439 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:57.146408 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-76576b46b5-n4hkm" Jul 28 17:17:57.355106 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:57.355082 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-76576b46b5-n4hkm"] Jul 28 17:17:57.357314 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:17:57.357280 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode11a57c0_9013_4492_9722_fd0aa4bb2c92.slice/crio-e558eb05b6063ae11642f8db4fc4e491c604056531f6396a52f9e3ed92fac6a5 WatchSource:0}: Error finding container e558eb05b6063ae11642f8db4fc4e491c604056531f6396a52f9e3ed92fac6a5: Status 404 returned error can't find the container with id e558eb05b6063ae11642f8db4fc4e491c604056531f6396a52f9e3ed92fac6a5 Jul 28 17:17:58.030171 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:58.030135 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-76576b46b5-n4hkm" event={"ID":"e11a57c0-9013-4492-9722-fd0aa4bb2c92","Type":"ContainerStarted","Data":"e558eb05b6063ae11642f8db4fc4e491c604056531f6396a52f9e3ed92fac6a5"} Jul 28 17:17:58.202759 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:58.202724 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-4n9g5"] Jul 28 17:17:58.206453 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:58.206430 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4n9g5" Jul 28 17:17:58.210283 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:58.210263 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 28 17:17:58.240665 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:58.240633 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-4n9g5"] Jul 28 17:17:58.368655 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:58.368574 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2594\" (UniqueName: \"kubernetes.io/projected/3c4430fa-54ac-4a06-87fc-08555d52774b-kube-api-access-w2594\") pod \"payload-pre-processing-687dbb6cf6-4n9g5\" (UID: \"3c4430fa-54ac-4a06-87fc-08555d52774b\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4n9g5" Jul 28 17:17:58.368800 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:58.368696 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c4430fa-54ac-4a06-87fc-08555d52774b-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-4n9g5\" (UID: \"3c4430fa-54ac-4a06-87fc-08555d52774b\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4n9g5" Jul 28 17:17:58.470481 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:58.470440 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2594\" (UniqueName: \"kubernetes.io/projected/3c4430fa-54ac-4a06-87fc-08555d52774b-kube-api-access-w2594\") pod \"payload-pre-processing-687dbb6cf6-4n9g5\" (UID: \"3c4430fa-54ac-4a06-87fc-08555d52774b\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4n9g5" Jul 28 17:17:58.470686 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:58.470558 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c4430fa-54ac-4a06-87fc-08555d52774b-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-4n9g5\" (UID: \"3c4430fa-54ac-4a06-87fc-08555d52774b\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4n9g5" Jul 28 17:17:58.471365 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:58.471343 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c4430fa-54ac-4a06-87fc-08555d52774b-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-4n9g5\" (UID: \"3c4430fa-54ac-4a06-87fc-08555d52774b\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4n9g5" Jul 28 17:17:58.492589 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:58.492559 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2594\" (UniqueName: \"kubernetes.io/projected/3c4430fa-54ac-4a06-87fc-08555d52774b-kube-api-access-w2594\") pod \"payload-pre-processing-687dbb6cf6-4n9g5\" (UID: \"3c4430fa-54ac-4a06-87fc-08555d52774b\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4n9g5" Jul 28 17:17:58.517319 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:58.517273 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4n9g5" Jul 28 17:17:58.694797 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:17:58.694766 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-4n9g5"] Jul 28 17:17:59.058183 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:17:59.057375 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c4430fa_54ac_4a06_87fc_08555d52774b.slice/crio-373d1855a96b6ed0f4c0240c74cdb4f4ea801a4fab68378585d4951de135ae3b WatchSource:0}: Error finding container 373d1855a96b6ed0f4c0240c74cdb4f4ea801a4fab68378585d4951de135ae3b: Status 404 returned error can't find the container with id 373d1855a96b6ed0f4c0240c74cdb4f4ea801a4fab68378585d4951de135ae3b Jul 28 17:18:00.064266 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:00.064229 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4n9g5" event={"ID":"3c4430fa-54ac-4a06-87fc-08555d52774b","Type":"ContainerStarted","Data":"373d1855a96b6ed0f4c0240c74cdb4f4ea801a4fab68378585d4951de135ae3b"} Jul 28 17:18:00.067084 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:00.067053 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-76576b46b5-n4hkm" event={"ID":"e11a57c0-9013-4492-9722-fd0aa4bb2c92","Type":"ContainerStarted","Data":"2fb4be5e79226b509f210bd11ddb7fcd9fd63552403465faef0313ea921e6a0c"} Jul 28 17:18:00.067232 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:00.067219 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-76576b46b5-n4hkm" Jul 28 17:18:00.108388 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:00.108308 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-76576b46b5-n4hkm" podStartSLOduration=2.3471692969999998 podStartE2EDuration="4.108287181s" totalImagesPullingTime="1.761117884s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:17:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:18:00.106242839 +0000 UTC m=+578.715174210" watchObservedRunningTime="2026-07-28 17:18:00.108287181 +0000 UTC m=+578.717218553" Jul 28 17:18:02.077646 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:02.077611 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4n9g5" event={"ID":"3c4430fa-54ac-4a06-87fc-08555d52774b","Type":"ContainerStarted","Data":"5868e210678bab7922ab3ed1d47318ca74d517ada380978b5806063ebc9f1c27"} Jul 28 17:18:02.078028 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:02.077751 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4n9g5" Jul 28 17:18:02.116912 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:02.116854 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4n9g5" podStartSLOduration=2.095118377 podStartE2EDuration="4.116835114s" totalImagesPullingTime="2.021716737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:17:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:18:02.116786722 +0000 UTC m=+580.725718111" watchObservedRunningTime="2026-07-28 17:18:02.116835114 +0000 UTC m=+580.725766485" Jul 28 17:18:04.579296 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:04.579250 2570 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-vnfxw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 28 17:18:04.579746 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:04.579309 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" podUID="ecd3cc49-6ce7-4db3-a0d0-797b1600822b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 28 17:18:06.077456 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:06.077382 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-76576b46b5-n4hkm" Jul 28 17:18:06.100796 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:06.100766 2570 generic.go:356] "Generic (PLEG): container finished" podID="ecd3cc49-6ce7-4db3-a0d0-797b1600822b" containerID="44244b60769bbb4707ea896febc387ec5f84740b1f30208311ec5b00e79f83ee" exitCode=0 Jul 28 17:18:06.100796 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:06.100790 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" event={"ID":"ecd3cc49-6ce7-4db3-a0d0-797b1600822b","Type":"ContainerDied","Data":"44244b60769bbb4707ea896febc387ec5f84740b1f30208311ec5b00e79f83ee"} Jul 28 17:18:06.101182 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:06.101168 2570 scope.go:117] "RemoveContainer" containerID="44244b60769bbb4707ea896febc387ec5f84740b1f30208311ec5b00e79f83ee" Jul 28 17:18:06.102093 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:06.102068 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 17:18:06.102170 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:06.102150 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 17:18:06.102221 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:06.102182 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 17:18:07.108545 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:07.108506 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" event={"ID":"ecd3cc49-6ce7-4db3-a0d0-797b1600822b","Type":"ContainerStarted","Data":"3e7f0a9d1b39f7222e342d40fdce0c80c96518aaea056b85e7193ed127461c81"} Jul 28 17:18:07.442401 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:07.442361 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" Jul 28 17:18:07.442604 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:07.442483 2570 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-vnfxw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 28 17:18:07.442604 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:07.442523 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" podUID="ecd3cc49-6ce7-4db3-a0d0-797b1600822b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 28 17:18:08.442690 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:08.442647 2570 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-vnfxw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 28 17:18:08.443193 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:08.442728 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" podUID="ecd3cc49-6ce7-4db3-a0d0-797b1600822b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 28 17:18:09.442555 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:09.442516 2570 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-vnfxw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 28 17:18:09.442763 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:09.442585 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" podUID="ecd3cc49-6ce7-4db3-a0d0-797b1600822b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 28 17:18:10.442329 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:10.442291 2570 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-vnfxw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 28 17:18:10.442605 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:10.442577 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" podUID="ecd3cc49-6ce7-4db3-a0d0-797b1600822b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 28 17:18:11.445702 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:11.445671 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" Jul 28 17:18:11.446183 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:11.446018 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" Jul 28 17:18:11.446598 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:11.446580 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vnfxw" Jul 28 17:18:13.086003 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:13.085972 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4n9g5" Jul 28 17:18:21.838498 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:21.838464 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-slfqf_ecb82800-8cf0-4ef9-926c-155172fad7ff/console-operator/1.log" Jul 28 17:18:21.842851 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:21.842828 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-slfqf_ecb82800-8cf0-4ef9-926c-155172fad7ff/console-operator/1.log" Jul 28 17:18:33.455148 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:33.455096 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-k47ks"] Jul 28 17:18:33.461192 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:33.461169 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k47ks" Jul 28 17:18:33.483067 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:33.483038 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 28 17:18:33.483225 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:33.483087 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 28 17:18:33.486366 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:33.486347 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-jddxn\"" Jul 28 17:18:33.486513 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:33.486498 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 28 17:18:33.493309 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:33.493281 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-k47ks"] Jul 28 17:18:33.598719 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:33.598677 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fae7d027-f7a8-494a-b88f-316046e21f1b-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k47ks\" (UID: \"fae7d027-f7a8-494a-b88f-316046e21f1b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k47ks" Jul 28 17:18:33.598918 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:33.598845 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86v9c\" (UniqueName: \"kubernetes.io/projected/fae7d027-f7a8-494a-b88f-316046e21f1b-kube-api-access-86v9c\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k47ks\" (UID: \"fae7d027-f7a8-494a-b88f-316046e21f1b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k47ks" Jul 28 17:18:33.599010 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:33.598949 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fae7d027-f7a8-494a-b88f-316046e21f1b-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k47ks\" (UID: \"fae7d027-f7a8-494a-b88f-316046e21f1b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k47ks" Jul 28 17:18:33.599010 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:33.598977 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fae7d027-f7a8-494a-b88f-316046e21f1b-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k47ks\" (UID: \"fae7d027-f7a8-494a-b88f-316046e21f1b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k47ks" Jul 28 17:18:33.599010 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:33.599003 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fae7d027-f7a8-494a-b88f-316046e21f1b-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k47ks\" (UID: \"fae7d027-f7a8-494a-b88f-316046e21f1b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k47ks" Jul 28 17:18:33.599167 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:33.599067 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fae7d027-f7a8-494a-b88f-316046e21f1b-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k47ks\" (UID: \"fae7d027-f7a8-494a-b88f-316046e21f1b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k47ks" Jul 28 17:18:33.699959 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:33.699920 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86v9c\" (UniqueName: \"kubernetes.io/projected/fae7d027-f7a8-494a-b88f-316046e21f1b-kube-api-access-86v9c\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k47ks\" (UID: \"fae7d027-f7a8-494a-b88f-316046e21f1b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k47ks" Jul 28 17:18:33.700177 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:33.699985 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fae7d027-f7a8-494a-b88f-316046e21f1b-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k47ks\" (UID: \"fae7d027-f7a8-494a-b88f-316046e21f1b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k47ks" Jul 28 17:18:33.700177 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:33.700002 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fae7d027-f7a8-494a-b88f-316046e21f1b-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k47ks\" (UID: \"fae7d027-f7a8-494a-b88f-316046e21f1b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k47ks" Jul 28 17:18:33.700177 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:33.700019 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fae7d027-f7a8-494a-b88f-316046e21f1b-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k47ks\" (UID: \"fae7d027-f7a8-494a-b88f-316046e21f1b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k47ks" Jul 28 17:18:33.700367 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:33.700189 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fae7d027-f7a8-494a-b88f-316046e21f1b-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k47ks\" (UID: \"fae7d027-f7a8-494a-b88f-316046e21f1b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k47ks" Jul 28 17:18:33.700367 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:33.700254 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fae7d027-f7a8-494a-b88f-316046e21f1b-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k47ks\" (UID: \"fae7d027-f7a8-494a-b88f-316046e21f1b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k47ks" Jul 28 17:18:33.700478 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:33.700459 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fae7d027-f7a8-494a-b88f-316046e21f1b-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k47ks\" (UID: \"fae7d027-f7a8-494a-b88f-316046e21f1b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k47ks" Jul 28 17:18:33.700527 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:33.700503 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fae7d027-f7a8-494a-b88f-316046e21f1b-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k47ks\" (UID: \"fae7d027-f7a8-494a-b88f-316046e21f1b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k47ks" Jul 28 17:18:33.700572 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:33.700557 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fae7d027-f7a8-494a-b88f-316046e21f1b-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k47ks\" (UID: \"fae7d027-f7a8-494a-b88f-316046e21f1b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k47ks" Jul 28 17:18:33.702435 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:33.702405 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fae7d027-f7a8-494a-b88f-316046e21f1b-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k47ks\" (UID: \"fae7d027-f7a8-494a-b88f-316046e21f1b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k47ks" Jul 28 17:18:33.702540 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:33.702453 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fae7d027-f7a8-494a-b88f-316046e21f1b-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k47ks\" (UID: \"fae7d027-f7a8-494a-b88f-316046e21f1b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k47ks" Jul 28 17:18:33.714957 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:33.714904 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86v9c\" (UniqueName: \"kubernetes.io/projected/fae7d027-f7a8-494a-b88f-316046e21f1b-kube-api-access-86v9c\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k47ks\" (UID: \"fae7d027-f7a8-494a-b88f-316046e21f1b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k47ks" Jul 28 17:18:33.770793 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:33.770753 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k47ks" Jul 28 17:18:33.931130 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:33.931087 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-k47ks"] Jul 28 17:18:33.932739 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:18:33.932706 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfae7d027_f7a8_494a_b88f_316046e21f1b.slice/crio-0a4db926face9675e463290d1325466335f2123e2ca91fcfdb817489aef66306 WatchSource:0}: Error finding container 0a4db926face9675e463290d1325466335f2123e2ca91fcfdb817489aef66306: Status 404 returned error can't find the container with id 0a4db926face9675e463290d1325466335f2123e2ca91fcfdb817489aef66306 Jul 28 17:18:34.254665 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:34.254624 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k47ks" event={"ID":"fae7d027-f7a8-494a-b88f-316046e21f1b","Type":"ContainerStarted","Data":"0a4db926face9675e463290d1325466335f2123e2ca91fcfdb817489aef66306"} Jul 28 17:18:35.261006 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:35.260928 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k47ks" event={"ID":"fae7d027-f7a8-494a-b88f-316046e21f1b","Type":"ContainerStarted","Data":"9d47162de5d2755f42ce09118a8a77b820e76c8419fbefce8b11485723e7ee0b"} Jul 28 17:18:35.324489 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:35.324437 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k47ks" podStartSLOduration=1.24736834 podStartE2EDuration="2.324419645s" totalImagesPullingTime="1.077051305s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:18:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:18:35.32203276 +0000 UTC m=+613.930964131" watchObservedRunningTime="2026-07-28 17:18:35.324419645 +0000 UTC m=+613.933351015" Jul 28 17:18:43.324718 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:43.324684 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67"] Jul 28 17:18:43.329159 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:43.329134 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67" Jul 28 17:18:43.332344 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:43.332315 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 28 17:18:43.350085 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:43.350050 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67"] Jul 28 17:18:43.492002 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:43.491950 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/958be73c-2e93-409e-92f0-e4013e60c958-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67\" (UID: \"958be73c-2e93-409e-92f0-e4013e60c958\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67" Jul 28 17:18:43.492218 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:43.492066 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/958be73c-2e93-409e-92f0-e4013e60c958-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67\" (UID: \"958be73c-2e93-409e-92f0-e4013e60c958\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67" Jul 28 17:18:43.492218 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:43.492101 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/958be73c-2e93-409e-92f0-e4013e60c958-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67\" (UID: \"958be73c-2e93-409e-92f0-e4013e60c958\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67" Jul 28 17:18:43.492218 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:43.492145 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5wfb\" (UniqueName: \"kubernetes.io/projected/958be73c-2e93-409e-92f0-e4013e60c958-kube-api-access-g5wfb\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67\" (UID: \"958be73c-2e93-409e-92f0-e4013e60c958\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67" Jul 28 17:18:43.492333 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:43.492218 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/958be73c-2e93-409e-92f0-e4013e60c958-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67\" (UID: \"958be73c-2e93-409e-92f0-e4013e60c958\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67" Jul 28 17:18:43.492333 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:43.492260 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/958be73c-2e93-409e-92f0-e4013e60c958-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67\" (UID: \"958be73c-2e93-409e-92f0-e4013e60c958\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67" Jul 28 17:18:43.593503 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:43.593416 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/958be73c-2e93-409e-92f0-e4013e60c958-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67\" (UID: \"958be73c-2e93-409e-92f0-e4013e60c958\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67" Jul 28 17:18:43.593503 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:43.593451 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/958be73c-2e93-409e-92f0-e4013e60c958-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67\" (UID: \"958be73c-2e93-409e-92f0-e4013e60c958\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67" Jul 28 17:18:43.593503 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:43.593468 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5wfb\" (UniqueName: \"kubernetes.io/projected/958be73c-2e93-409e-92f0-e4013e60c958-kube-api-access-g5wfb\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67\" (UID: \"958be73c-2e93-409e-92f0-e4013e60c958\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67" Jul 28 17:18:43.593789 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:43.593588 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/958be73c-2e93-409e-92f0-e4013e60c958-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67\" (UID: \"958be73c-2e93-409e-92f0-e4013e60c958\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67" Jul 28 17:18:43.593789 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:43.593649 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/958be73c-2e93-409e-92f0-e4013e60c958-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67\" (UID: \"958be73c-2e93-409e-92f0-e4013e60c958\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67" Jul 28 17:18:43.593789 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:43.593726 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/958be73c-2e93-409e-92f0-e4013e60c958-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67\" (UID: \"958be73c-2e93-409e-92f0-e4013e60c958\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67" Jul 28 17:18:43.593937 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:43.593868 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/958be73c-2e93-409e-92f0-e4013e60c958-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67\" (UID: \"958be73c-2e93-409e-92f0-e4013e60c958\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67" Jul 28 17:18:43.593937 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:43.593884 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/958be73c-2e93-409e-92f0-e4013e60c958-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67\" (UID: \"958be73c-2e93-409e-92f0-e4013e60c958\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67" Jul 28 17:18:43.594004 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:43.593989 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/958be73c-2e93-409e-92f0-e4013e60c958-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67\" (UID: \"958be73c-2e93-409e-92f0-e4013e60c958\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67" Jul 28 17:18:43.595845 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:43.595811 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/958be73c-2e93-409e-92f0-e4013e60c958-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67\" (UID: \"958be73c-2e93-409e-92f0-e4013e60c958\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67" Jul 28 17:18:43.596031 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:43.596017 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/958be73c-2e93-409e-92f0-e4013e60c958-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67\" (UID: \"958be73c-2e93-409e-92f0-e4013e60c958\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67" Jul 28 17:18:43.614342 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:43.614319 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5wfb\" (UniqueName: \"kubernetes.io/projected/958be73c-2e93-409e-92f0-e4013e60c958-kube-api-access-g5wfb\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67\" (UID: \"958be73c-2e93-409e-92f0-e4013e60c958\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67" Jul 28 17:18:43.641512 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:43.641480 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67" Jul 28 17:18:43.770903 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:43.770847 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k47ks" Jul 28 17:18:43.770903 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:43.770901 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k47ks" Jul 28 17:18:43.783086 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:43.783059 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k47ks" Jul 28 17:18:43.803803 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:43.803782 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67"] Jul 28 17:18:43.805810 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:18:43.805782 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod958be73c_2e93_409e_92f0_e4013e60c958.slice/crio-9c544c1f6b29d5bfc04b98559ed48b2ace168e6ff577bd697caa6087833ae129 WatchSource:0}: Error finding container 9c544c1f6b29d5bfc04b98559ed48b2ace168e6ff577bd697caa6087833ae129: Status 404 returned error can't find the container with id 9c544c1f6b29d5bfc04b98559ed48b2ace168e6ff577bd697caa6087833ae129 Jul 28 17:18:44.308224 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:44.308129 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67" event={"ID":"958be73c-2e93-409e-92f0-e4013e60c958","Type":"ContainerStarted","Data":"b77be3def571215a9681ddd6e2641517d186a8b1338b93760843537aacc1f758"} Jul 28 17:18:44.308224 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:44.308194 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67" event={"ID":"958be73c-2e93-409e-92f0-e4013e60c958","Type":"ContainerStarted","Data":"9c544c1f6b29d5bfc04b98559ed48b2ace168e6ff577bd697caa6087833ae129"} Jul 28 17:18:44.319339 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:44.319316 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k47ks" Jul 28 17:18:44.359127 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:44.359076 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67" podStartSLOduration=1.021505724 podStartE2EDuration="1.359061991s" totalImagesPullingTime="337.556267ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:18:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:18:44.337603235 +0000 UTC m=+622.946534606" watchObservedRunningTime="2026-07-28 17:18:44.359061991 +0000 UTC m=+622.967993361" Jul 28 17:18:53.642225 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:53.642181 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67" Jul 28 17:18:53.642621 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:53.642522 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67" Jul 28 17:18:53.654929 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:53.654898 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67" Jul 28 17:18:54.371440 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:54.371413 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67" Jul 28 17:18:55.902053 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:55.902022 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl"] Jul 28 17:18:55.905617 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:55.905597 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl" Jul 28 17:18:55.909487 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:55.909464 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 28 17:18:55.919978 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:55.919959 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl"] Jul 28 17:18:55.997429 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:55.997389 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4c06325-fc6a-473a-812a-b6f72bd4dfc9-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl\" (UID: \"a4c06325-fc6a-473a-812a-b6f72bd4dfc9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl" Jul 28 17:18:55.997607 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:55.997449 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5z57\" (UniqueName: \"kubernetes.io/projected/a4c06325-fc6a-473a-812a-b6f72bd4dfc9-kube-api-access-w5z57\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl\" (UID: \"a4c06325-fc6a-473a-812a-b6f72bd4dfc9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl" Jul 28 17:18:55.997607 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:55.997495 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4c06325-fc6a-473a-812a-b6f72bd4dfc9-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl\" (UID: \"a4c06325-fc6a-473a-812a-b6f72bd4dfc9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl" Jul 28 17:18:55.997607 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:55.997573 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4c06325-fc6a-473a-812a-b6f72bd4dfc9-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl\" (UID: \"a4c06325-fc6a-473a-812a-b6f72bd4dfc9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl" Jul 28 17:18:55.997719 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:55.997630 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4c06325-fc6a-473a-812a-b6f72bd4dfc9-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl\" (UID: \"a4c06325-fc6a-473a-812a-b6f72bd4dfc9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl" Jul 28 17:18:55.997719 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:55.997692 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4c06325-fc6a-473a-812a-b6f72bd4dfc9-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl\" (UID: \"a4c06325-fc6a-473a-812a-b6f72bd4dfc9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl" Jul 28 17:18:56.099040 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:56.099008 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4c06325-fc6a-473a-812a-b6f72bd4dfc9-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl\" (UID: \"a4c06325-fc6a-473a-812a-b6f72bd4dfc9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl" Jul 28 17:18:56.099040 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:56.099046 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4c06325-fc6a-473a-812a-b6f72bd4dfc9-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl\" (UID: \"a4c06325-fc6a-473a-812a-b6f72bd4dfc9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl" Jul 28 17:18:56.099284 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:56.099185 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4c06325-fc6a-473a-812a-b6f72bd4dfc9-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl\" (UID: \"a4c06325-fc6a-473a-812a-b6f72bd4dfc9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl" Jul 28 17:18:56.099284 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:56.099253 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4c06325-fc6a-473a-812a-b6f72bd4dfc9-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl\" (UID: \"a4c06325-fc6a-473a-812a-b6f72bd4dfc9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl" Jul 28 17:18:56.099284 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:56.099278 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5z57\" (UniqueName: \"kubernetes.io/projected/a4c06325-fc6a-473a-812a-b6f72bd4dfc9-kube-api-access-w5z57\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl\" (UID: \"a4c06325-fc6a-473a-812a-b6f72bd4dfc9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl" Jul 28 17:18:56.099395 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:56.099308 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4c06325-fc6a-473a-812a-b6f72bd4dfc9-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl\" (UID: \"a4c06325-fc6a-473a-812a-b6f72bd4dfc9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl" Jul 28 17:18:56.099490 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:56.099470 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4c06325-fc6a-473a-812a-b6f72bd4dfc9-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl\" (UID: \"a4c06325-fc6a-473a-812a-b6f72bd4dfc9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl" Jul 28 17:18:56.099559 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:56.099525 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4c06325-fc6a-473a-812a-b6f72bd4dfc9-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl\" (UID: \"a4c06325-fc6a-473a-812a-b6f72bd4dfc9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl" Jul 28 17:18:56.099655 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:56.099631 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4c06325-fc6a-473a-812a-b6f72bd4dfc9-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl\" (UID: \"a4c06325-fc6a-473a-812a-b6f72bd4dfc9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl" Jul 28 17:18:56.101508 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:56.101475 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4c06325-fc6a-473a-812a-b6f72bd4dfc9-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl\" (UID: \"a4c06325-fc6a-473a-812a-b6f72bd4dfc9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl" Jul 28 17:18:56.101783 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:56.101765 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4c06325-fc6a-473a-812a-b6f72bd4dfc9-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl\" (UID: \"a4c06325-fc6a-473a-812a-b6f72bd4dfc9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl" Jul 28 17:18:56.110157 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:56.110127 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5z57\" (UniqueName: \"kubernetes.io/projected/a4c06325-fc6a-473a-812a-b6f72bd4dfc9-kube-api-access-w5z57\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl\" (UID: \"a4c06325-fc6a-473a-812a-b6f72bd4dfc9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl" Jul 28 17:18:56.215974 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:56.215887 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl" Jul 28 17:18:56.373184 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:56.373151 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl"] Jul 28 17:18:56.374520 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:18:56.374487 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda4c06325_fc6a_473a_812a_b6f72bd4dfc9.slice/crio-9f378e0df1a2b8acc90fc3726f8a7ddd2e504529992c335c54adf176228d4829 WatchSource:0}: Error finding container 9f378e0df1a2b8acc90fc3726f8a7ddd2e504529992c335c54adf176228d4829: Status 404 returned error can't find the container with id 9f378e0df1a2b8acc90fc3726f8a7ddd2e504529992c335c54adf176228d4829 Jul 28 17:18:56.375858 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:56.375841 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 17:18:57.376292 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:57.376254 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl" event={"ID":"a4c06325-fc6a-473a-812a-b6f72bd4dfc9","Type":"ContainerStarted","Data":"2cbcbf5f97d91cea7a076e321443fef787d27e35698a7aa126e8246b11d58c66"} Jul 28 17:18:57.376654 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:57.376300 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl" event={"ID":"a4c06325-fc6a-473a-812a-b6f72bd4dfc9","Type":"ContainerStarted","Data":"9f378e0df1a2b8acc90fc3726f8a7ddd2e504529992c335c54adf176228d4829"} Jul 28 17:18:57.404746 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:18:57.404691 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl" podStartSLOduration=2.225423718 podStartE2EDuration="2.404675551s" totalImagesPullingTime="179.251833ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:18:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:18:57.402864761 +0000 UTC m=+636.011796131" watchObservedRunningTime="2026-07-28 17:18:57.404675551 +0000 UTC m=+636.013606921" Jul 28 17:19:06.217071 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:19:06.217036 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl" Jul 28 17:19:06.217488 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:19:06.217087 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl" Jul 28 17:19:06.229765 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:19:06.229737 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl" Jul 28 17:19:06.434068 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:19:06.434039 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl" Jul 28 17:19:46.538274 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:19:46.538235 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-84b4d8cb66-bnw9p"] Jul 28 17:19:46.542704 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:19:46.542681 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84b4d8cb66-bnw9p" Jul 28 17:19:46.553700 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:19:46.553672 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-84b4d8cb66-bnw9p"] Jul 28 17:19:46.588694 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:19:46.588657 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvn9x\" (UniqueName: \"kubernetes.io/projected/4b8835fa-ff72-4eb8-943e-fe808e69690b-kube-api-access-dvn9x\") pod \"authorino-84b4d8cb66-bnw9p\" (UID: \"4b8835fa-ff72-4eb8-943e-fe808e69690b\") " pod="kuadrant-system/authorino-84b4d8cb66-bnw9p" Jul 28 17:19:46.588857 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:19:46.588700 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4b8835fa-ff72-4eb8-943e-fe808e69690b-tls-cert\") pod \"authorino-84b4d8cb66-bnw9p\" (UID: \"4b8835fa-ff72-4eb8-943e-fe808e69690b\") " pod="kuadrant-system/authorino-84b4d8cb66-bnw9p" Jul 28 17:19:46.689318 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:19:46.689287 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvn9x\" (UniqueName: \"kubernetes.io/projected/4b8835fa-ff72-4eb8-943e-fe808e69690b-kube-api-access-dvn9x\") pod \"authorino-84b4d8cb66-bnw9p\" (UID: \"4b8835fa-ff72-4eb8-943e-fe808e69690b\") " pod="kuadrant-system/authorino-84b4d8cb66-bnw9p" Jul 28 17:19:46.689490 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:19:46.689327 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4b8835fa-ff72-4eb8-943e-fe808e69690b-tls-cert\") pod \"authorino-84b4d8cb66-bnw9p\" (UID: \"4b8835fa-ff72-4eb8-943e-fe808e69690b\") " pod="kuadrant-system/authorino-84b4d8cb66-bnw9p" Jul 28 17:19:46.691740 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:19:46.691715 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4b8835fa-ff72-4eb8-943e-fe808e69690b-tls-cert\") pod \"authorino-84b4d8cb66-bnw9p\" (UID: \"4b8835fa-ff72-4eb8-943e-fe808e69690b\") " pod="kuadrant-system/authorino-84b4d8cb66-bnw9p" Jul 28 17:19:46.700579 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:19:46.700556 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvn9x\" (UniqueName: \"kubernetes.io/projected/4b8835fa-ff72-4eb8-943e-fe808e69690b-kube-api-access-dvn9x\") pod \"authorino-84b4d8cb66-bnw9p\" (UID: \"4b8835fa-ff72-4eb8-943e-fe808e69690b\") " pod="kuadrant-system/authorino-84b4d8cb66-bnw9p" Jul 28 17:19:46.852782 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:19:46.852702 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84b4d8cb66-bnw9p" Jul 28 17:19:46.996635 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:19:46.996609 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-84b4d8cb66-bnw9p"] Jul 28 17:19:46.998889 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:19:46.998865 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b8835fa_ff72_4eb8_943e_fe808e69690b.slice/crio-ae444feef6dcffe6d6cd158f36f690f14ce7826252fe9568872b933cd8a3563b WatchSource:0}: Error finding container ae444feef6dcffe6d6cd158f36f690f14ce7826252fe9568872b933cd8a3563b: Status 404 returned error can't find the container with id ae444feef6dcffe6d6cd158f36f690f14ce7826252fe9568872b933cd8a3563b Jul 28 17:19:47.640309 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:19:47.640271 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84b4d8cb66-bnw9p" event={"ID":"4b8835fa-ff72-4eb8-943e-fe808e69690b","Type":"ContainerStarted","Data":"78d50bd7ac61702a84fbc56483c777863c9d3ee36e67cbdbd18845a3bbe55f2a"} Jul 28 17:19:47.640309 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:19:47.640311 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84b4d8cb66-bnw9p" event={"ID":"4b8835fa-ff72-4eb8-943e-fe808e69690b","Type":"ContainerStarted","Data":"ae444feef6dcffe6d6cd158f36f690f14ce7826252fe9568872b933cd8a3563b"} Jul 28 17:19:47.665374 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:19:47.665310 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-84b4d8cb66-bnw9p" podStartSLOduration=1.204576825 podStartE2EDuration="1.665294896s" totalImagesPullingTime="460.718071ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:19:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:19:47.665059013 +0000 UTC m=+686.273990383" watchObservedRunningTime="2026-07-28 17:19:47.665294896 +0000 UTC m=+686.274226265" Jul 28 17:19:47.717162 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:19:47.717040 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-d8df5c7b7-c28tl"] Jul 28 17:19:47.717349 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:19:47.717235 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-d8df5c7b7-c28tl" podUID="028cacb5-b9d3-402f-8a34-56621ebded34" containerName="authorino" containerID="cri-o://088977583abe04045bf81b6956de34a1a2f0482d4e377273679a296344b1a2a4" gracePeriod=30 Jul 28 17:19:48.006018 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:19:48.005671 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d8df5c7b7-c28tl" Jul 28 17:19:48.102471 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:19:48.102442 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sf4v9\" (UniqueName: \"kubernetes.io/projected/028cacb5-b9d3-402f-8a34-56621ebded34-kube-api-access-sf4v9\") pod \"028cacb5-b9d3-402f-8a34-56621ebded34\" (UID: \"028cacb5-b9d3-402f-8a34-56621ebded34\") " Jul 28 17:19:48.102641 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:19:48.102490 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/028cacb5-b9d3-402f-8a34-56621ebded34-tls-cert\") pod \"028cacb5-b9d3-402f-8a34-56621ebded34\" (UID: \"028cacb5-b9d3-402f-8a34-56621ebded34\") " Jul 28 17:19:48.104703 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:19:48.104669 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/028cacb5-b9d3-402f-8a34-56621ebded34-kube-api-access-sf4v9" (OuterVolumeSpecName: "kube-api-access-sf4v9") pod "028cacb5-b9d3-402f-8a34-56621ebded34" (UID: "028cacb5-b9d3-402f-8a34-56621ebded34"). InnerVolumeSpecName "kube-api-access-sf4v9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:19:48.114386 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:19:48.114362 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/028cacb5-b9d3-402f-8a34-56621ebded34-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "028cacb5-b9d3-402f-8a34-56621ebded34" (UID: "028cacb5-b9d3-402f-8a34-56621ebded34"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:19:48.204355 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:19:48.204308 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sf4v9\" (UniqueName: \"kubernetes.io/projected/028cacb5-b9d3-402f-8a34-56621ebded34-kube-api-access-sf4v9\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:19:48.204355 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:19:48.204351 2570 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/028cacb5-b9d3-402f-8a34-56621ebded34-tls-cert\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:19:48.648329 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:19:48.648292 2570 generic.go:356] "Generic (PLEG): container finished" podID="028cacb5-b9d3-402f-8a34-56621ebded34" containerID="088977583abe04045bf81b6956de34a1a2f0482d4e377273679a296344b1a2a4" exitCode=0 Jul 28 17:19:48.648811 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:19:48.648351 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d8df5c7b7-c28tl" Jul 28 17:19:48.648811 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:19:48.648380 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d8df5c7b7-c28tl" event={"ID":"028cacb5-b9d3-402f-8a34-56621ebded34","Type":"ContainerDied","Data":"088977583abe04045bf81b6956de34a1a2f0482d4e377273679a296344b1a2a4"} Jul 28 17:19:48.648811 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:19:48.648437 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d8df5c7b7-c28tl" event={"ID":"028cacb5-b9d3-402f-8a34-56621ebded34","Type":"ContainerDied","Data":"6d441b1644f5a8683ff2d20ed05e6e0f60249901d2fbbcda76d8b8722de2541a"} Jul 28 17:19:48.648811 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:19:48.648462 2570 scope.go:117] "RemoveContainer" containerID="088977583abe04045bf81b6956de34a1a2f0482d4e377273679a296344b1a2a4" Jul 28 17:19:48.658482 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:19:48.658460 2570 scope.go:117] "RemoveContainer" containerID="088977583abe04045bf81b6956de34a1a2f0482d4e377273679a296344b1a2a4" Jul 28 17:19:48.658765 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:19:48.658740 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"088977583abe04045bf81b6956de34a1a2f0482d4e377273679a296344b1a2a4\": container with ID starting with 088977583abe04045bf81b6956de34a1a2f0482d4e377273679a296344b1a2a4 not found: ID does not exist" containerID="088977583abe04045bf81b6956de34a1a2f0482d4e377273679a296344b1a2a4" Jul 28 17:19:48.658853 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:19:48.658776 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"088977583abe04045bf81b6956de34a1a2f0482d4e377273679a296344b1a2a4"} err="failed to get container status \"088977583abe04045bf81b6956de34a1a2f0482d4e377273679a296344b1a2a4\": rpc error: code = NotFound desc = could not find container \"088977583abe04045bf81b6956de34a1a2f0482d4e377273679a296344b1a2a4\": container with ID starting with 088977583abe04045bf81b6956de34a1a2f0482d4e377273679a296344b1a2a4 not found: ID does not exist" Jul 28 17:19:48.672223 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:19:48.672192 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-d8df5c7b7-c28tl"] Jul 28 17:19:48.680714 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:19:48.680683 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-d8df5c7b7-c28tl"] Jul 28 17:19:50.121158 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:19:50.121131 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="028cacb5-b9d3-402f-8a34-56621ebded34" path="/var/lib/kubelet/pods/028cacb5-b9d3-402f-8a34-56621ebded34/volumes" Jul 28 17:20:38.874677 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:20:38.874634 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7d6f646b66-vlsbm"] Jul 28 17:20:38.875229 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:20:38.874881 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7d6f646b66-vlsbm" podUID="517748fc-2967-4965-8247-a82c806cee67" containerName="manager" containerID="cri-o://1f1ddab83675ff70484710921498be73d43e1bebea3ee5c170a9a22482b190f3" gracePeriod=30 Jul 28 17:20:39.128886 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:20:39.128831 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d6f646b66-vlsbm" Jul 28 17:20:39.191677 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:20:39.191646 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9d25\" (UniqueName: \"kubernetes.io/projected/517748fc-2967-4965-8247-a82c806cee67-kube-api-access-d9d25\") pod \"517748fc-2967-4965-8247-a82c806cee67\" (UID: \"517748fc-2967-4965-8247-a82c806cee67\") " Jul 28 17:20:39.191860 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:20:39.191707 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/517748fc-2967-4965-8247-a82c806cee67-webhook-cert\") pod \"517748fc-2967-4965-8247-a82c806cee67\" (UID: \"517748fc-2967-4965-8247-a82c806cee67\") " Jul 28 17:20:39.193657 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:20:39.193628 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/517748fc-2967-4965-8247-a82c806cee67-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "517748fc-2967-4965-8247-a82c806cee67" (UID: "517748fc-2967-4965-8247-a82c806cee67"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:20:39.193771 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:20:39.193668 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/517748fc-2967-4965-8247-a82c806cee67-kube-api-access-d9d25" (OuterVolumeSpecName: "kube-api-access-d9d25") pod "517748fc-2967-4965-8247-a82c806cee67" (UID: "517748fc-2967-4965-8247-a82c806cee67"). InnerVolumeSpecName "kube-api-access-d9d25". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:20:39.292872 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:20:39.292833 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9d25\" (UniqueName: \"kubernetes.io/projected/517748fc-2967-4965-8247-a82c806cee67-kube-api-access-d9d25\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:20:39.292872 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:20:39.292869 2570 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/517748fc-2967-4965-8247-a82c806cee67-webhook-cert\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:20:39.927781 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:20:39.927740 2570 generic.go:356] "Generic (PLEG): container finished" podID="517748fc-2967-4965-8247-a82c806cee67" containerID="1f1ddab83675ff70484710921498be73d43e1bebea3ee5c170a9a22482b190f3" exitCode=0 Jul 28 17:20:39.928208 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:20:39.927779 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d6f646b66-vlsbm" event={"ID":"517748fc-2967-4965-8247-a82c806cee67","Type":"ContainerDied","Data":"1f1ddab83675ff70484710921498be73d43e1bebea3ee5c170a9a22482b190f3"} Jul 28 17:20:39.928208 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:20:39.927799 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d6f646b66-vlsbm" Jul 28 17:20:39.928208 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:20:39.927863 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d6f646b66-vlsbm" event={"ID":"517748fc-2967-4965-8247-a82c806cee67","Type":"ContainerDied","Data":"a511d0610204db5ac69a27dd80afc27f3864ab69a9fba6241b589c9879ab2677"} Jul 28 17:20:39.928208 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:20:39.927887 2570 scope.go:117] "RemoveContainer" containerID="1f1ddab83675ff70484710921498be73d43e1bebea3ee5c170a9a22482b190f3" Jul 28 17:20:39.938019 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:20:39.938004 2570 scope.go:117] "RemoveContainer" containerID="1f1ddab83675ff70484710921498be73d43e1bebea3ee5c170a9a22482b190f3" Jul 28 17:20:39.938316 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:20:39.938296 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f1ddab83675ff70484710921498be73d43e1bebea3ee5c170a9a22482b190f3\": container with ID starting with 1f1ddab83675ff70484710921498be73d43e1bebea3ee5c170a9a22482b190f3 not found: ID does not exist" containerID="1f1ddab83675ff70484710921498be73d43e1bebea3ee5c170a9a22482b190f3" Jul 28 17:20:39.938392 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:20:39.938327 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f1ddab83675ff70484710921498be73d43e1bebea3ee5c170a9a22482b190f3"} err="failed to get container status \"1f1ddab83675ff70484710921498be73d43e1bebea3ee5c170a9a22482b190f3\": rpc error: code = NotFound desc = could not find container \"1f1ddab83675ff70484710921498be73d43e1bebea3ee5c170a9a22482b190f3\": container with ID starting with 1f1ddab83675ff70484710921498be73d43e1bebea3ee5c170a9a22482b190f3 not found: ID does not exist" Jul 28 17:20:39.951217 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:20:39.951192 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7d6f646b66-vlsbm"] Jul 28 17:20:39.958282 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:20:39.958261 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7d6f646b66-vlsbm"] Jul 28 17:20:40.120105 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:20:40.120068 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="517748fc-2967-4965-8247-a82c806cee67" path="/var/lib/kubelet/pods/517748fc-2967-4965-8247-a82c806cee67/volumes" Jul 28 17:21:40.787943 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:40.787907 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7d6f646b66-7r298"] Jul 28 17:21:40.788602 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:40.788524 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="028cacb5-b9d3-402f-8a34-56621ebded34" containerName="authorino" Jul 28 17:21:40.788602 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:40.788546 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="028cacb5-b9d3-402f-8a34-56621ebded34" containerName="authorino" Jul 28 17:21:40.788602 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:40.788588 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="517748fc-2967-4965-8247-a82c806cee67" containerName="manager" Jul 28 17:21:40.788602 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:40.788597 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="517748fc-2967-4965-8247-a82c806cee67" containerName="manager" Jul 28 17:21:40.788824 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:40.788680 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="028cacb5-b9d3-402f-8a34-56621ebded34" containerName="authorino" Jul 28 17:21:40.788824 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:40.788695 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="517748fc-2967-4965-8247-a82c806cee67" containerName="manager" Jul 28 17:21:40.797982 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:40.797948 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d6f646b66-7r298" Jul 28 17:21:40.800748 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:40.800713 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 28 17:21:40.801094 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:40.801074 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 28 17:21:40.801226 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:40.801074 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-z2rlk\"" Jul 28 17:21:40.802147 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:40.802107 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d6f646b66-7r298"] Jul 28 17:21:40.855600 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:40.855569 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgqjh\" (UniqueName: \"kubernetes.io/projected/79f41070-13d7-408f-8263-7e957598d9c8-kube-api-access-wgqjh\") pod \"maas-controller-7d6f646b66-7r298\" (UID: \"79f41070-13d7-408f-8263-7e957598d9c8\") " pod="opendatahub/maas-controller-7d6f646b66-7r298" Jul 28 17:21:40.855746 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:40.855627 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/79f41070-13d7-408f-8263-7e957598d9c8-webhook-cert\") pod \"maas-controller-7d6f646b66-7r298\" (UID: \"79f41070-13d7-408f-8263-7e957598d9c8\") " pod="opendatahub/maas-controller-7d6f646b66-7r298" Jul 28 17:21:40.956427 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:40.956394 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgqjh\" (UniqueName: \"kubernetes.io/projected/79f41070-13d7-408f-8263-7e957598d9c8-kube-api-access-wgqjh\") pod \"maas-controller-7d6f646b66-7r298\" (UID: \"79f41070-13d7-408f-8263-7e957598d9c8\") " pod="opendatahub/maas-controller-7d6f646b66-7r298" Jul 28 17:21:40.956609 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:40.956450 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/79f41070-13d7-408f-8263-7e957598d9c8-webhook-cert\") pod \"maas-controller-7d6f646b66-7r298\" (UID: \"79f41070-13d7-408f-8263-7e957598d9c8\") " pod="opendatahub/maas-controller-7d6f646b66-7r298" Jul 28 17:21:40.959045 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:40.959011 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/79f41070-13d7-408f-8263-7e957598d9c8-webhook-cert\") pod \"maas-controller-7d6f646b66-7r298\" (UID: \"79f41070-13d7-408f-8263-7e957598d9c8\") " pod="opendatahub/maas-controller-7d6f646b66-7r298" Jul 28 17:21:40.965195 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:40.965165 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgqjh\" (UniqueName: \"kubernetes.io/projected/79f41070-13d7-408f-8263-7e957598d9c8-kube-api-access-wgqjh\") pod \"maas-controller-7d6f646b66-7r298\" (UID: \"79f41070-13d7-408f-8263-7e957598d9c8\") " pod="opendatahub/maas-controller-7d6f646b66-7r298" Jul 28 17:21:41.109933 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:41.109842 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d6f646b66-7r298" Jul 28 17:21:41.247422 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:41.247398 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d6f646b66-7r298"] Jul 28 17:21:41.249491 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:21:41.249463 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79f41070_13d7_408f_8263_7e957598d9c8.slice/crio-aacfcd5fc396393b819b0d85c3dc7c057cbc158566012555b62f28c0df1d0ad9 WatchSource:0}: Error finding container aacfcd5fc396393b819b0d85c3dc7c057cbc158566012555b62f28c0df1d0ad9: Status 404 returned error can't find the container with id aacfcd5fc396393b819b0d85c3dc7c057cbc158566012555b62f28c0df1d0ad9 Jul 28 17:21:41.261777 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:41.261751 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d6f646b66-7r298" event={"ID":"79f41070-13d7-408f-8263-7e957598d9c8","Type":"ContainerStarted","Data":"aacfcd5fc396393b819b0d85c3dc7c057cbc158566012555b62f28c0df1d0ad9"} Jul 28 17:21:42.268643 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:42.268591 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d6f646b66-7r298" event={"ID":"79f41070-13d7-408f-8263-7e957598d9c8","Type":"ContainerStarted","Data":"20040d4939a1cfa25d489770e9326f1b1dfc50fa5b9928332685ac8b12f959e0"} Jul 28 17:21:42.269086 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:42.268665 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7d6f646b66-7r298" Jul 28 17:21:42.289361 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:42.289307 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7d6f646b66-7r298" podStartSLOduration=1.771723041 podStartE2EDuration="2.289291723s" totalImagesPullingTime="517.568682ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:21:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:21:42.286894518 +0000 UTC m=+800.895825899" watchObservedRunningTime="2026-07-28 17:21:42.289291723 +0000 UTC m=+800.898223093" Jul 28 17:21:53.276815 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:53.276736 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7d6f646b66-7r298" Jul 28 17:21:57.953589 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:57.953545 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7d6f646b66-7r298"] Jul 28 17:21:57.954058 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:57.953792 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7d6f646b66-7r298" podUID="79f41070-13d7-408f-8263-7e957598d9c8" containerName="manager" containerID="cri-o://20040d4939a1cfa25d489770e9326f1b1dfc50fa5b9928332685ac8b12f959e0" gracePeriod=30 Jul 28 17:21:58.215447 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:58.215384 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d6f646b66-7r298" Jul 28 17:21:58.315383 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:58.315352 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wgqjh\" (UniqueName: \"kubernetes.io/projected/79f41070-13d7-408f-8263-7e957598d9c8-kube-api-access-wgqjh\") pod \"79f41070-13d7-408f-8263-7e957598d9c8\" (UID: \"79f41070-13d7-408f-8263-7e957598d9c8\") " Jul 28 17:21:58.315568 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:58.315407 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/79f41070-13d7-408f-8263-7e957598d9c8-webhook-cert\") pod \"79f41070-13d7-408f-8263-7e957598d9c8\" (UID: \"79f41070-13d7-408f-8263-7e957598d9c8\") " Jul 28 17:21:58.317455 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:58.317424 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79f41070-13d7-408f-8263-7e957598d9c8-kube-api-access-wgqjh" (OuterVolumeSpecName: "kube-api-access-wgqjh") pod "79f41070-13d7-408f-8263-7e957598d9c8" (UID: "79f41070-13d7-408f-8263-7e957598d9c8"). InnerVolumeSpecName "kube-api-access-wgqjh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:21:58.317558 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:58.317470 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79f41070-13d7-408f-8263-7e957598d9c8-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "79f41070-13d7-408f-8263-7e957598d9c8" (UID: "79f41070-13d7-408f-8263-7e957598d9c8"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:21:58.354427 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:58.354391 2570 generic.go:356] "Generic (PLEG): container finished" podID="79f41070-13d7-408f-8263-7e957598d9c8" containerID="20040d4939a1cfa25d489770e9326f1b1dfc50fa5b9928332685ac8b12f959e0" exitCode=0 Jul 28 17:21:58.354586 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:58.354446 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d6f646b66-7r298" event={"ID":"79f41070-13d7-408f-8263-7e957598d9c8","Type":"ContainerDied","Data":"20040d4939a1cfa25d489770e9326f1b1dfc50fa5b9928332685ac8b12f959e0"} Jul 28 17:21:58.354586 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:58.354510 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d6f646b66-7r298" event={"ID":"79f41070-13d7-408f-8263-7e957598d9c8","Type":"ContainerDied","Data":"aacfcd5fc396393b819b0d85c3dc7c057cbc158566012555b62f28c0df1d0ad9"} Jul 28 17:21:58.354586 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:58.354449 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d6f646b66-7r298" Jul 28 17:21:58.354586 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:58.354530 2570 scope.go:117] "RemoveContainer" containerID="20040d4939a1cfa25d489770e9326f1b1dfc50fa5b9928332685ac8b12f959e0" Jul 28 17:21:58.365662 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:58.365640 2570 scope.go:117] "RemoveContainer" containerID="20040d4939a1cfa25d489770e9326f1b1dfc50fa5b9928332685ac8b12f959e0" Jul 28 17:21:58.366095 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:21:58.366029 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20040d4939a1cfa25d489770e9326f1b1dfc50fa5b9928332685ac8b12f959e0\": container with ID starting with 20040d4939a1cfa25d489770e9326f1b1dfc50fa5b9928332685ac8b12f959e0 not found: ID does not exist" containerID="20040d4939a1cfa25d489770e9326f1b1dfc50fa5b9928332685ac8b12f959e0" Jul 28 17:21:58.366095 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:58.366061 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20040d4939a1cfa25d489770e9326f1b1dfc50fa5b9928332685ac8b12f959e0"} err="failed to get container status \"20040d4939a1cfa25d489770e9326f1b1dfc50fa5b9928332685ac8b12f959e0\": rpc error: code = NotFound desc = could not find container \"20040d4939a1cfa25d489770e9326f1b1dfc50fa5b9928332685ac8b12f959e0\": container with ID starting with 20040d4939a1cfa25d489770e9326f1b1dfc50fa5b9928332685ac8b12f959e0 not found: ID does not exist" Jul 28 17:21:58.379492 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:58.379458 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7d6f646b66-7r298"] Jul 28 17:21:58.383346 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:58.383322 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7d6f646b66-7r298"] Jul 28 17:21:58.416327 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:58.416303 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wgqjh\" (UniqueName: \"kubernetes.io/projected/79f41070-13d7-408f-8263-7e957598d9c8-kube-api-access-wgqjh\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:21:58.416327 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:58.416324 2570 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/79f41070-13d7-408f-8263-7e957598d9c8-webhook-cert\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:21:59.185203 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:59.185170 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7d6f646b66-jww2t"] Jul 28 17:21:59.185599 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:59.185585 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79f41070-13d7-408f-8263-7e957598d9c8" containerName="manager" Jul 28 17:21:59.185644 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:59.185602 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="79f41070-13d7-408f-8263-7e957598d9c8" containerName="manager" Jul 28 17:21:59.185679 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:59.185670 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="79f41070-13d7-408f-8263-7e957598d9c8" containerName="manager" Jul 28 17:21:59.189966 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:59.189947 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d6f646b66-jww2t" Jul 28 17:21:59.192814 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:59.192797 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 28 17:21:59.193799 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:59.193780 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 28 17:21:59.193888 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:59.193803 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-z2rlk\"" Jul 28 17:21:59.197335 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:59.197149 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d6f646b66-jww2t"] Jul 28 17:21:59.325596 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:59.325540 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwmmw\" (UniqueName: \"kubernetes.io/projected/4facf865-9753-4107-82ef-fd66a0bb9e93-kube-api-access-fwmmw\") pod \"maas-controller-7d6f646b66-jww2t\" (UID: \"4facf865-9753-4107-82ef-fd66a0bb9e93\") " pod="opendatahub/maas-controller-7d6f646b66-jww2t" Jul 28 17:21:59.325763 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:59.325630 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4facf865-9753-4107-82ef-fd66a0bb9e93-webhook-cert\") pod \"maas-controller-7d6f646b66-jww2t\" (UID: \"4facf865-9753-4107-82ef-fd66a0bb9e93\") " pod="opendatahub/maas-controller-7d6f646b66-jww2t" Jul 28 17:21:59.426769 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:59.426731 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4facf865-9753-4107-82ef-fd66a0bb9e93-webhook-cert\") pod \"maas-controller-7d6f646b66-jww2t\" (UID: \"4facf865-9753-4107-82ef-fd66a0bb9e93\") " pod="opendatahub/maas-controller-7d6f646b66-jww2t" Jul 28 17:21:59.426944 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:59.426820 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwmmw\" (UniqueName: \"kubernetes.io/projected/4facf865-9753-4107-82ef-fd66a0bb9e93-kube-api-access-fwmmw\") pod \"maas-controller-7d6f646b66-jww2t\" (UID: \"4facf865-9753-4107-82ef-fd66a0bb9e93\") " pod="opendatahub/maas-controller-7d6f646b66-jww2t" Jul 28 17:21:59.429169 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:59.429143 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4facf865-9753-4107-82ef-fd66a0bb9e93-webhook-cert\") pod \"maas-controller-7d6f646b66-jww2t\" (UID: \"4facf865-9753-4107-82ef-fd66a0bb9e93\") " pod="opendatahub/maas-controller-7d6f646b66-jww2t" Jul 28 17:21:59.437673 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:59.437605 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwmmw\" (UniqueName: \"kubernetes.io/projected/4facf865-9753-4107-82ef-fd66a0bb9e93-kube-api-access-fwmmw\") pod \"maas-controller-7d6f646b66-jww2t\" (UID: \"4facf865-9753-4107-82ef-fd66a0bb9e93\") " pod="opendatahub/maas-controller-7d6f646b66-jww2t" Jul 28 17:21:59.502106 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:59.502070 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d6f646b66-jww2t" Jul 28 17:21:59.655915 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:21:59.655893 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d6f646b66-jww2t"] Jul 28 17:21:59.658331 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:21:59.658305 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4facf865_9753_4107_82ef_fd66a0bb9e93.slice/crio-5ddec60affb72818ab40104fc227583288184ebc69c18ff24db9978db4b9620e WatchSource:0}: Error finding container 5ddec60affb72818ab40104fc227583288184ebc69c18ff24db9978db4b9620e: Status 404 returned error can't find the container with id 5ddec60affb72818ab40104fc227583288184ebc69c18ff24db9978db4b9620e Jul 28 17:22:00.119820 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:22:00.119787 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79f41070-13d7-408f-8263-7e957598d9c8" path="/var/lib/kubelet/pods/79f41070-13d7-408f-8263-7e957598d9c8/volumes" Jul 28 17:22:00.367049 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:22:00.366961 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d6f646b66-jww2t" event={"ID":"4facf865-9753-4107-82ef-fd66a0bb9e93","Type":"ContainerStarted","Data":"7f9892fa3b512bb304bffca621f1cabe436d5d1089522cbbadd53498d3f566a9"} Jul 28 17:22:00.367049 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:22:00.367018 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d6f646b66-jww2t" event={"ID":"4facf865-9753-4107-82ef-fd66a0bb9e93","Type":"ContainerStarted","Data":"5ddec60affb72818ab40104fc227583288184ebc69c18ff24db9978db4b9620e"} Jul 28 17:22:00.367497 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:22:00.367086 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7d6f646b66-jww2t" Jul 28 17:22:00.386372 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:22:00.386323 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7d6f646b66-jww2t" podStartSLOduration=1.09574059 podStartE2EDuration="1.386307378s" totalImagesPullingTime="290.566788ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:21:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:22:00.385127483 +0000 UTC m=+818.994058851" watchObservedRunningTime="2026-07-28 17:22:00.386307378 +0000 UTC m=+818.995238747" Jul 28 17:22:11.374123 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:22:11.374090 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7d6f646b66-jww2t" Jul 28 17:23:21.879273 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:23:21.879243 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-slfqf_ecb82800-8cf0-4ef9-926c-155172fad7ff/console-operator/1.log" Jul 28 17:23:21.881667 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:23:21.881641 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-slfqf_ecb82800-8cf0-4ef9-926c-155172fad7ff/console-operator/1.log" Jul 28 17:26:42.663757 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:26:42.663723 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvgcx"] Jul 28 17:26:42.664384 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:26:42.663915 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvgcx" podUID="5988cae6-f955-4969-8996-1b9c663e596e" containerName="manager" containerID="cri-o://f85e987e5e69275eb7a62ca48f99c0be295c47132cc43cd54647dadd5ad92226" gracePeriod=10 Jul 28 17:26:46.443014 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:26:46.442965 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvgcx" podUID="5988cae6-f955-4969-8996-1b9c663e596e" containerName="manager" probeResult="failure" output="Get \"http://10.132.0.41:8081/readyz\": dial tcp 10.132.0.41:8081: connect: connection refused" Jul 28 17:26:46.865574 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:26:46.865543 2570 generic.go:356] "Generic (PLEG): container finished" podID="5988cae6-f955-4969-8996-1b9c663e596e" containerID="f85e987e5e69275eb7a62ca48f99c0be295c47132cc43cd54647dadd5ad92226" exitCode=0 Jul 28 17:26:46.865724 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:26:46.865610 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvgcx" event={"ID":"5988cae6-f955-4969-8996-1b9c663e596e","Type":"ContainerDied","Data":"f85e987e5e69275eb7a62ca48f99c0be295c47132cc43cd54647dadd5ad92226"} Jul 28 17:26:46.918827 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:26:46.918801 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvgcx" Jul 28 17:26:47.036932 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:26:47.036846 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98ctn\" (UniqueName: \"kubernetes.io/projected/5988cae6-f955-4969-8996-1b9c663e596e-kube-api-access-98ctn\") pod \"5988cae6-f955-4969-8996-1b9c663e596e\" (UID: \"5988cae6-f955-4969-8996-1b9c663e596e\") " Jul 28 17:26:47.036932 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:26:47.036928 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5988cae6-f955-4969-8996-1b9c663e596e-extensions-socket-volume\") pod \"5988cae6-f955-4969-8996-1b9c663e596e\" (UID: \"5988cae6-f955-4969-8996-1b9c663e596e\") " Jul 28 17:26:47.037351 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:26:47.037321 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5988cae6-f955-4969-8996-1b9c663e596e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5988cae6-f955-4969-8996-1b9c663e596e" (UID: "5988cae6-f955-4969-8996-1b9c663e596e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:26:47.038992 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:26:47.038966 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5988cae6-f955-4969-8996-1b9c663e596e-kube-api-access-98ctn" (OuterVolumeSpecName: "kube-api-access-98ctn") pod "5988cae6-f955-4969-8996-1b9c663e596e" (UID: "5988cae6-f955-4969-8996-1b9c663e596e"). InnerVolumeSpecName "kube-api-access-98ctn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:26:47.138448 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:26:47.138412 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98ctn\" (UniqueName: \"kubernetes.io/projected/5988cae6-f955-4969-8996-1b9c663e596e-kube-api-access-98ctn\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:26:47.138448 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:26:47.138442 2570 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5988cae6-f955-4969-8996-1b9c663e596e-extensions-socket-volume\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:26:47.871662 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:26:47.871626 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvgcx" event={"ID":"5988cae6-f955-4969-8996-1b9c663e596e","Type":"ContainerDied","Data":"74b3524561e20dd7bf6407a1f724baec6fc539fb6b200bf195e17f7b9dcba0f7"} Jul 28 17:26:47.871662 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:26:47.871646 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvgcx" Jul 28 17:26:47.872163 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:26:47.871688 2570 scope.go:117] "RemoveContainer" containerID="f85e987e5e69275eb7a62ca48f99c0be295c47132cc43cd54647dadd5ad92226" Jul 28 17:26:47.900240 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:26:47.900213 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvgcx"] Jul 28 17:26:47.907979 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:26:47.907955 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvgcx"] Jul 28 17:26:48.120295 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:26:48.120262 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5988cae6-f955-4969-8996-1b9c663e596e" path="/var/lib/kubelet/pods/5988cae6-f955-4969-8996-1b9c663e596e/volumes" Jul 28 17:27:43.869219 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:27:43.869182 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ds7ns"] Jul 28 17:27:43.869612 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:27:43.869584 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5988cae6-f955-4969-8996-1b9c663e596e" containerName="manager" Jul 28 17:27:43.869612 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:27:43.869594 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="5988cae6-f955-4969-8996-1b9c663e596e" containerName="manager" Jul 28 17:27:43.869686 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:27:43.869659 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="5988cae6-f955-4969-8996-1b9c663e596e" containerName="manager" Jul 28 17:27:43.873841 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:27:43.873823 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ds7ns" Jul 28 17:27:43.877730 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:27:43.877712 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-pb6mp\"" Jul 28 17:27:43.888539 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:27:43.888512 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ds7ns"] Jul 28 17:27:43.955791 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:27:43.955760 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9l8mg\" (UniqueName: \"kubernetes.io/projected/27bc78bb-e7d4-4796-9d00-7c9454975d57-kube-api-access-9l8mg\") pod \"kuadrant-operator-controller-manager-86878c769f-ds7ns\" (UID: \"27bc78bb-e7d4-4796-9d00-7c9454975d57\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ds7ns" Jul 28 17:27:43.955947 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:27:43.955819 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/27bc78bb-e7d4-4796-9d00-7c9454975d57-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ds7ns\" (UID: \"27bc78bb-e7d4-4796-9d00-7c9454975d57\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ds7ns" Jul 28 17:27:44.056608 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:27:44.056575 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l8mg\" (UniqueName: \"kubernetes.io/projected/27bc78bb-e7d4-4796-9d00-7c9454975d57-kube-api-access-9l8mg\") pod \"kuadrant-operator-controller-manager-86878c769f-ds7ns\" (UID: \"27bc78bb-e7d4-4796-9d00-7c9454975d57\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ds7ns" Jul 28 17:27:44.056754 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:27:44.056626 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/27bc78bb-e7d4-4796-9d00-7c9454975d57-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ds7ns\" (UID: \"27bc78bb-e7d4-4796-9d00-7c9454975d57\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ds7ns" Jul 28 17:27:44.056995 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:27:44.056978 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/27bc78bb-e7d4-4796-9d00-7c9454975d57-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ds7ns\" (UID: \"27bc78bb-e7d4-4796-9d00-7c9454975d57\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ds7ns" Jul 28 17:27:44.073733 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:27:44.073703 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9l8mg\" (UniqueName: \"kubernetes.io/projected/27bc78bb-e7d4-4796-9d00-7c9454975d57-kube-api-access-9l8mg\") pod \"kuadrant-operator-controller-manager-86878c769f-ds7ns\" (UID: \"27bc78bb-e7d4-4796-9d00-7c9454975d57\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ds7ns" Jul 28 17:27:44.183784 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:27:44.183752 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ds7ns" Jul 28 17:27:44.329506 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:27:44.329474 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ds7ns"] Jul 28 17:27:44.330541 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:27:44.330510 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27bc78bb_e7d4_4796_9d00_7c9454975d57.slice/crio-97d0935d26864624c499685e6f4975c08f91d8414b6aee290a01ce3a0d2198d0 WatchSource:0}: Error finding container 97d0935d26864624c499685e6f4975c08f91d8414b6aee290a01ce3a0d2198d0: Status 404 returned error can't find the container with id 97d0935d26864624c499685e6f4975c08f91d8414b6aee290a01ce3a0d2198d0 Jul 28 17:27:45.168262 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:27:45.168230 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ds7ns" event={"ID":"27bc78bb-e7d4-4796-9d00-7c9454975d57","Type":"ContainerStarted","Data":"a33151e56906329ffd66677993af4cca7d0bfd8a322273e496cd903c44cab059"} Jul 28 17:27:45.168663 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:27:45.168275 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ds7ns" event={"ID":"27bc78bb-e7d4-4796-9d00-7c9454975d57","Type":"ContainerStarted","Data":"97d0935d26864624c499685e6f4975c08f91d8414b6aee290a01ce3a0d2198d0"} Jul 28 17:27:45.168663 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:27:45.168370 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ds7ns" Jul 28 17:27:45.207398 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:27:45.207354 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ds7ns" podStartSLOduration=2.207341285 podStartE2EDuration="2.207341285s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:27:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:27:45.205350978 +0000 UTC m=+1163.814282349" watchObservedRunningTime="2026-07-28 17:27:45.207341285 +0000 UTC m=+1163.816272654" Jul 28 17:27:56.175247 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:27:56.175216 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ds7ns" Jul 28 17:28:21.912835 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:28:21.912806 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-slfqf_ecb82800-8cf0-4ef9-926c-155172fad7ff/console-operator/1.log" Jul 28 17:28:21.915281 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:28:21.912806 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-slfqf_ecb82800-8cf0-4ef9-926c-155172fad7ff/console-operator/1.log" Jul 28 17:33:21.944714 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:33:21.944604 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-slfqf_ecb82800-8cf0-4ef9-926c-155172fad7ff/console-operator/1.log" Jul 28 17:33:21.948936 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:33:21.945912 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-slfqf_ecb82800-8cf0-4ef9-926c-155172fad7ff/console-operator/1.log" Jul 28 17:34:53.369570 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:53.369537 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl"] Jul 28 17:34:53.373592 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:53.373574 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" Jul 28 17:34:53.376311 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:53.376288 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-949b0a76-openshift-default-dockercfg-nr2mv\"" Jul 28 17:34:53.396023 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:53.396000 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl"] Jul 28 17:34:53.529945 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:53.529905 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c614212-e197-44f3-84a3-e6f38a10aaef-workload-certs\") pod \"e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl\" (UID: \"2c614212-e197-44f3-84a3-e6f38a10aaef\") " pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" Jul 28 17:34:53.529945 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:53.529958 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c614212-e197-44f3-84a3-e6f38a10aaef-istio-envoy\") pod \"e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl\" (UID: \"2c614212-e197-44f3-84a3-e6f38a10aaef\") " pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" Jul 28 17:34:53.530222 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:53.530015 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c614212-e197-44f3-84a3-e6f38a10aaef-workload-socket\") pod \"e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl\" (UID: \"2c614212-e197-44f3-84a3-e6f38a10aaef\") " pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" Jul 28 17:34:53.530222 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:53.530136 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c614212-e197-44f3-84a3-e6f38a10aaef-istio-token\") pod \"e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl\" (UID: \"2c614212-e197-44f3-84a3-e6f38a10aaef\") " pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" Jul 28 17:34:53.530222 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:53.530178 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c614212-e197-44f3-84a3-e6f38a10aaef-istiod-ca-cert\") pod \"e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl\" (UID: \"2c614212-e197-44f3-84a3-e6f38a10aaef\") " pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" Jul 28 17:34:53.530222 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:53.530204 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c614212-e197-44f3-84a3-e6f38a10aaef-credential-socket\") pod \"e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl\" (UID: \"2c614212-e197-44f3-84a3-e6f38a10aaef\") " pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" Jul 28 17:34:53.530373 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:53.530249 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c614212-e197-44f3-84a3-e6f38a10aaef-istio-data\") pod \"e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl\" (UID: \"2c614212-e197-44f3-84a3-e6f38a10aaef\") " pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" Jul 28 17:34:53.530373 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:53.530270 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c614212-e197-44f3-84a3-e6f38a10aaef-istio-podinfo\") pod \"e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl\" (UID: \"2c614212-e197-44f3-84a3-e6f38a10aaef\") " pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" Jul 28 17:34:53.530373 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:53.530296 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8d65j\" (UniqueName: \"kubernetes.io/projected/2c614212-e197-44f3-84a3-e6f38a10aaef-kube-api-access-8d65j\") pod \"e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl\" (UID: \"2c614212-e197-44f3-84a3-e6f38a10aaef\") " pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" Jul 28 17:34:53.631084 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:53.630987 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c614212-e197-44f3-84a3-e6f38a10aaef-workload-certs\") pod \"e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl\" (UID: \"2c614212-e197-44f3-84a3-e6f38a10aaef\") " pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" Jul 28 17:34:53.631084 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:53.631026 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c614212-e197-44f3-84a3-e6f38a10aaef-istio-envoy\") pod \"e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl\" (UID: \"2c614212-e197-44f3-84a3-e6f38a10aaef\") " pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" Jul 28 17:34:53.631084 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:53.631051 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c614212-e197-44f3-84a3-e6f38a10aaef-workload-socket\") pod \"e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl\" (UID: \"2c614212-e197-44f3-84a3-e6f38a10aaef\") " pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" Jul 28 17:34:53.631084 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:53.631069 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c614212-e197-44f3-84a3-e6f38a10aaef-istio-token\") pod \"e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl\" (UID: \"2c614212-e197-44f3-84a3-e6f38a10aaef\") " pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" Jul 28 17:34:53.631442 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:53.631158 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c614212-e197-44f3-84a3-e6f38a10aaef-istiod-ca-cert\") pod \"e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl\" (UID: \"2c614212-e197-44f3-84a3-e6f38a10aaef\") " pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" Jul 28 17:34:53.631442 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:53.631194 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c614212-e197-44f3-84a3-e6f38a10aaef-credential-socket\") pod \"e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl\" (UID: \"2c614212-e197-44f3-84a3-e6f38a10aaef\") " pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" Jul 28 17:34:53.631442 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:53.631252 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c614212-e197-44f3-84a3-e6f38a10aaef-istio-data\") pod \"e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl\" (UID: \"2c614212-e197-44f3-84a3-e6f38a10aaef\") " pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" Jul 28 17:34:53.631442 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:53.631285 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c614212-e197-44f3-84a3-e6f38a10aaef-istio-podinfo\") pod \"e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl\" (UID: \"2c614212-e197-44f3-84a3-e6f38a10aaef\") " pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" Jul 28 17:34:53.631442 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:53.631323 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8d65j\" (UniqueName: \"kubernetes.io/projected/2c614212-e197-44f3-84a3-e6f38a10aaef-kube-api-access-8d65j\") pod \"e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl\" (UID: \"2c614212-e197-44f3-84a3-e6f38a10aaef\") " pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" Jul 28 17:34:53.631694 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:53.631494 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c614212-e197-44f3-84a3-e6f38a10aaef-workload-socket\") pod \"e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl\" (UID: \"2c614212-e197-44f3-84a3-e6f38a10aaef\") " pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" Jul 28 17:34:53.631694 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:53.631669 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c614212-e197-44f3-84a3-e6f38a10aaef-workload-certs\") pod \"e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl\" (UID: \"2c614212-e197-44f3-84a3-e6f38a10aaef\") " pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" Jul 28 17:34:53.631806 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:53.631741 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c614212-e197-44f3-84a3-e6f38a10aaef-istio-data\") pod \"e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl\" (UID: \"2c614212-e197-44f3-84a3-e6f38a10aaef\") " pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" Jul 28 17:34:53.631903 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:53.631805 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c614212-e197-44f3-84a3-e6f38a10aaef-credential-socket\") pod \"e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl\" (UID: \"2c614212-e197-44f3-84a3-e6f38a10aaef\") " pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" Jul 28 17:34:53.631964 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:53.631945 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c614212-e197-44f3-84a3-e6f38a10aaef-istiod-ca-cert\") pod \"e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl\" (UID: \"2c614212-e197-44f3-84a3-e6f38a10aaef\") " pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" Jul 28 17:34:53.633865 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:53.633836 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c614212-e197-44f3-84a3-e6f38a10aaef-istio-podinfo\") pod \"e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl\" (UID: \"2c614212-e197-44f3-84a3-e6f38a10aaef\") " pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" Jul 28 17:34:53.633993 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:53.633836 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c614212-e197-44f3-84a3-e6f38a10aaef-istio-envoy\") pod \"e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl\" (UID: \"2c614212-e197-44f3-84a3-e6f38a10aaef\") " pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" Jul 28 17:34:53.641248 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:53.641224 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d65j\" (UniqueName: \"kubernetes.io/projected/2c614212-e197-44f3-84a3-e6f38a10aaef-kube-api-access-8d65j\") pod \"e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl\" (UID: \"2c614212-e197-44f3-84a3-e6f38a10aaef\") " pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" Jul 28 17:34:53.641602 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:53.641584 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c614212-e197-44f3-84a3-e6f38a10aaef-istio-token\") pod \"e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl\" (UID: \"2c614212-e197-44f3-84a3-e6f38a10aaef\") " pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" Jul 28 17:34:53.684316 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:53.684280 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" Jul 28 17:34:53.831749 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:53.831720 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl"] Jul 28 17:34:53.833184 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:34:53.833150 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c614212_e197_44f3_84a3_e6f38a10aaef.slice/crio-437671ec19b63fe7fdc656fc5b04c757ef468fbc57cfaaf4a538bd1e358dfc01 WatchSource:0}: Error finding container 437671ec19b63fe7fdc656fc5b04c757ef468fbc57cfaaf4a538bd1e358dfc01: Status 404 returned error can't find the container with id 437671ec19b63fe7fdc656fc5b04c757ef468fbc57cfaaf4a538bd1e358dfc01 Jul 28 17:34:53.835489 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:53.835451 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 17:34:53.835576 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:53.835539 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 17:34:53.835613 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:53.835579 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 17:34:54.414715 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:54.414675 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" event={"ID":"2c614212-e197-44f3-84a3-e6f38a10aaef","Type":"ContainerStarted","Data":"0fd6060ed3c8ce26aaa6f12ed87f756875627777eb4e718b6e90a3c504d698ac"} Jul 28 17:34:54.415139 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:54.414741 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" event={"ID":"2c614212-e197-44f3-84a3-e6f38a10aaef","Type":"ContainerStarted","Data":"437671ec19b63fe7fdc656fc5b04c757ef468fbc57cfaaf4a538bd1e358dfc01"} Jul 28 17:34:54.440284 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:54.440227 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" podStartSLOduration=1.440212514 podStartE2EDuration="1.440212514s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:34:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:34:54.438269364 +0000 UTC m=+1593.047200734" watchObservedRunningTime="2026-07-28 17:34:54.440212514 +0000 UTC m=+1593.049143949" Jul 28 17:34:54.684721 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:54.684642 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" Jul 28 17:34:54.689330 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:54.689307 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" Jul 28 17:34:55.419742 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:55.419714 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" Jul 28 17:34:55.420847 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:34:55.420828 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" Jul 28 17:35:11.392095 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:11.390864 2570 patch_prober.go:28] interesting pod/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": read tcp 10.132.0.2:58466->10.132.0.61:15021: read: connection reset by peer" start-of-body= Jul 28 17:35:11.392095 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:11.390940 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" podUID="2c614212-e197-44f3-84a3-e6f38a10aaef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": read tcp 10.132.0.2:58466->10.132.0.61:15021: read: connection reset by peer" Jul 28 17:35:11.506536 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:11.506501 2570 generic.go:356] "Generic (PLEG): container finished" podID="2c614212-e197-44f3-84a3-e6f38a10aaef" containerID="0fd6060ed3c8ce26aaa6f12ed87f756875627777eb4e718b6e90a3c504d698ac" exitCode=0 Jul 28 17:35:11.506536 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:11.506541 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" event={"ID":"2c614212-e197-44f3-84a3-e6f38a10aaef","Type":"ContainerDied","Data":"0fd6060ed3c8ce26aaa6f12ed87f756875627777eb4e718b6e90a3c504d698ac"} Jul 28 17:35:11.506924 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:11.506909 2570 scope.go:117] "RemoveContainer" containerID="0fd6060ed3c8ce26aaa6f12ed87f756875627777eb4e718b6e90a3c504d698ac" Jul 28 17:35:11.507966 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:11.507936 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 17:35:11.508035 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:11.508023 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 17:35:11.508071 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:11.508063 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 17:35:12.513846 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:12.513810 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" event={"ID":"2c614212-e197-44f3-84a3-e6f38a10aaef","Type":"ContainerStarted","Data":"40ff459cc47e5bef5adbd3d93270e676e7caf0dd7e43a71a7df8e157c923a7ec"} Jul 28 17:35:12.685224 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:12.685184 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" Jul 28 17:35:12.685437 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:12.685420 2570 patch_prober.go:28] interesting pod/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 28 17:35:12.685493 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:12.685473 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" podUID="2c614212-e197-44f3-84a3-e6f38a10aaef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 28 17:35:13.685081 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:13.685037 2570 patch_prober.go:28] interesting pod/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 28 17:35:13.685574 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:13.685155 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" podUID="2c614212-e197-44f3-84a3-e6f38a10aaef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 28 17:35:14.685601 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:14.685513 2570 patch_prober.go:28] interesting pod/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 28 17:35:14.685601 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:14.685567 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" podUID="2c614212-e197-44f3-84a3-e6f38a10aaef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 28 17:35:15.684942 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:15.684906 2570 patch_prober.go:28] interesting pod/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 28 17:35:15.685134 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:15.684957 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" podUID="2c614212-e197-44f3-84a3-e6f38a10aaef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 28 17:35:16.688427 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:16.688389 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" Jul 28 17:35:16.688839 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:16.688698 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" Jul 28 17:35:16.689428 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:16.689408 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" Jul 28 17:35:36.744378 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:36.744344 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl"] Jul 28 17:35:36.744914 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:36.744573 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" podUID="2c614212-e197-44f3-84a3-e6f38a10aaef" containerName="istio-proxy" containerID="cri-o://40ff459cc47e5bef5adbd3d93270e676e7caf0dd7e43a71a7df8e157c923a7ec" gracePeriod=30 Jul 28 17:35:42.000480 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.000455 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" Jul 28 17:35:42.110462 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.110367 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c614212-e197-44f3-84a3-e6f38a10aaef-istio-token\") pod \"2c614212-e197-44f3-84a3-e6f38a10aaef\" (UID: \"2c614212-e197-44f3-84a3-e6f38a10aaef\") " Jul 28 17:35:42.110462 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.110431 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8d65j\" (UniqueName: \"kubernetes.io/projected/2c614212-e197-44f3-84a3-e6f38a10aaef-kube-api-access-8d65j\") pod \"2c614212-e197-44f3-84a3-e6f38a10aaef\" (UID: \"2c614212-e197-44f3-84a3-e6f38a10aaef\") " Jul 28 17:35:42.110672 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.110485 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c614212-e197-44f3-84a3-e6f38a10aaef-workload-certs\") pod \"2c614212-e197-44f3-84a3-e6f38a10aaef\" (UID: \"2c614212-e197-44f3-84a3-e6f38a10aaef\") " Jul 28 17:35:42.110672 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.110527 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c614212-e197-44f3-84a3-e6f38a10aaef-credential-socket\") pod \"2c614212-e197-44f3-84a3-e6f38a10aaef\" (UID: \"2c614212-e197-44f3-84a3-e6f38a10aaef\") " Jul 28 17:35:42.110672 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.110645 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c614212-e197-44f3-84a3-e6f38a10aaef-istiod-ca-cert\") pod \"2c614212-e197-44f3-84a3-e6f38a10aaef\" (UID: \"2c614212-e197-44f3-84a3-e6f38a10aaef\") " Jul 28 17:35:42.110832 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.110725 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c614212-e197-44f3-84a3-e6f38a10aaef-workload-socket\") pod \"2c614212-e197-44f3-84a3-e6f38a10aaef\" (UID: \"2c614212-e197-44f3-84a3-e6f38a10aaef\") " Jul 28 17:35:42.110832 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.110753 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c614212-e197-44f3-84a3-e6f38a10aaef-istio-data\") pod \"2c614212-e197-44f3-84a3-e6f38a10aaef\" (UID: \"2c614212-e197-44f3-84a3-e6f38a10aaef\") " Jul 28 17:35:42.110832 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.110780 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c614212-e197-44f3-84a3-e6f38a10aaef-istio-podinfo\") pod \"2c614212-e197-44f3-84a3-e6f38a10aaef\" (UID: \"2c614212-e197-44f3-84a3-e6f38a10aaef\") " Jul 28 17:35:42.110983 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.110834 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c614212-e197-44f3-84a3-e6f38a10aaef-istio-envoy\") pod \"2c614212-e197-44f3-84a3-e6f38a10aaef\" (UID: \"2c614212-e197-44f3-84a3-e6f38a10aaef\") " Jul 28 17:35:42.110983 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.110840 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c614212-e197-44f3-84a3-e6f38a10aaef-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2c614212-e197-44f3-84a3-e6f38a10aaef" (UID: "2c614212-e197-44f3-84a3-e6f38a10aaef"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:35:42.110983 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.110876 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c614212-e197-44f3-84a3-e6f38a10aaef-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2c614212-e197-44f3-84a3-e6f38a10aaef" (UID: "2c614212-e197-44f3-84a3-e6f38a10aaef"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:35:42.111172 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.111053 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c614212-e197-44f3-84a3-e6f38a10aaef-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2c614212-e197-44f3-84a3-e6f38a10aaef" (UID: "2c614212-e197-44f3-84a3-e6f38a10aaef"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:35:42.111172 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.111080 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c614212-e197-44f3-84a3-e6f38a10aaef-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2c614212-e197-44f3-84a3-e6f38a10aaef" (UID: "2c614212-e197-44f3-84a3-e6f38a10aaef"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:35:42.111295 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.111218 2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c614212-e197-44f3-84a3-e6f38a10aaef-credential-socket\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:35:42.111295 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.111236 2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c614212-e197-44f3-84a3-e6f38a10aaef-istiod-ca-cert\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:35:42.111295 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.111249 2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c614212-e197-44f3-84a3-e6f38a10aaef-workload-socket\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:35:42.111295 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.111261 2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c614212-e197-44f3-84a3-e6f38a10aaef-workload-certs\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:35:42.111677 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.111660 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c614212-e197-44f3-84a3-e6f38a10aaef-istio-data" (OuterVolumeSpecName: "istio-data") pod "2c614212-e197-44f3-84a3-e6f38a10aaef" (UID: "2c614212-e197-44f3-84a3-e6f38a10aaef"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:35:42.113172 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.113145 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c614212-e197-44f3-84a3-e6f38a10aaef-istio-token" (OuterVolumeSpecName: "istio-token") pod "2c614212-e197-44f3-84a3-e6f38a10aaef" (UID: "2c614212-e197-44f3-84a3-e6f38a10aaef"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:35:42.113297 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.113213 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c614212-e197-44f3-84a3-e6f38a10aaef-kube-api-access-8d65j" (OuterVolumeSpecName: "kube-api-access-8d65j") pod "2c614212-e197-44f3-84a3-e6f38a10aaef" (UID: "2c614212-e197-44f3-84a3-e6f38a10aaef"). InnerVolumeSpecName "kube-api-access-8d65j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:35:42.113297 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.113236 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2c614212-e197-44f3-84a3-e6f38a10aaef-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2c614212-e197-44f3-84a3-e6f38a10aaef" (UID: "2c614212-e197-44f3-84a3-e6f38a10aaef"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 28 17:35:42.113403 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.113321 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c614212-e197-44f3-84a3-e6f38a10aaef-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2c614212-e197-44f3-84a3-e6f38a10aaef" (UID: "2c614212-e197-44f3-84a3-e6f38a10aaef"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:35:42.212163 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.212107 2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c614212-e197-44f3-84a3-e6f38a10aaef-istio-data\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:35:42.212163 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.212161 2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c614212-e197-44f3-84a3-e6f38a10aaef-istio-podinfo\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:35:42.212163 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.212170 2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c614212-e197-44f3-84a3-e6f38a10aaef-istio-envoy\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:35:42.212391 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.212178 2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c614212-e197-44f3-84a3-e6f38a10aaef-istio-token\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:35:42.212391 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.212188 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8d65j\" (UniqueName: \"kubernetes.io/projected/2c614212-e197-44f3-84a3-e6f38a10aaef-kube-api-access-8d65j\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:35:42.674691 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.674657 2570 generic.go:356] "Generic (PLEG): container finished" podID="2c614212-e197-44f3-84a3-e6f38a10aaef" containerID="40ff459cc47e5bef5adbd3d93270e676e7caf0dd7e43a71a7df8e157c923a7ec" exitCode=0 Jul 28 17:35:42.674875 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.674725 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" Jul 28 17:35:42.674875 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.674745 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" event={"ID":"2c614212-e197-44f3-84a3-e6f38a10aaef","Type":"ContainerDied","Data":"40ff459cc47e5bef5adbd3d93270e676e7caf0dd7e43a71a7df8e157c923a7ec"} Jul 28 17:35:42.674875 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.674832 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl" event={"ID":"2c614212-e197-44f3-84a3-e6f38a10aaef","Type":"ContainerDied","Data":"437671ec19b63fe7fdc656fc5b04c757ef468fbc57cfaaf4a538bd1e358dfc01"} Jul 28 17:35:42.674875 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.674858 2570 scope.go:117] "RemoveContainer" containerID="40ff459cc47e5bef5adbd3d93270e676e7caf0dd7e43a71a7df8e157c923a7ec" Jul 28 17:35:42.683848 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.683830 2570 scope.go:117] "RemoveContainer" containerID="0fd6060ed3c8ce26aaa6f12ed87f756875627777eb4e718b6e90a3c504d698ac" Jul 28 17:35:42.691712 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.691689 2570 scope.go:117] "RemoveContainer" containerID="40ff459cc47e5bef5adbd3d93270e676e7caf0dd7e43a71a7df8e157c923a7ec" Jul 28 17:35:42.691987 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:35:42.691966 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40ff459cc47e5bef5adbd3d93270e676e7caf0dd7e43a71a7df8e157c923a7ec\": container with ID starting with 40ff459cc47e5bef5adbd3d93270e676e7caf0dd7e43a71a7df8e157c923a7ec not found: ID does not exist" containerID="40ff459cc47e5bef5adbd3d93270e676e7caf0dd7e43a71a7df8e157c923a7ec" Jul 28 17:35:42.692051 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.691996 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40ff459cc47e5bef5adbd3d93270e676e7caf0dd7e43a71a7df8e157c923a7ec"} err="failed to get container status \"40ff459cc47e5bef5adbd3d93270e676e7caf0dd7e43a71a7df8e157c923a7ec\": rpc error: code = NotFound desc = could not find container \"40ff459cc47e5bef5adbd3d93270e676e7caf0dd7e43a71a7df8e157c923a7ec\": container with ID starting with 40ff459cc47e5bef5adbd3d93270e676e7caf0dd7e43a71a7df8e157c923a7ec not found: ID does not exist" Jul 28 17:35:42.692051 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.692014 2570 scope.go:117] "RemoveContainer" containerID="0fd6060ed3c8ce26aaa6f12ed87f756875627777eb4e718b6e90a3c504d698ac" Jul 28 17:35:42.692307 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:35:42.692273 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fd6060ed3c8ce26aaa6f12ed87f756875627777eb4e718b6e90a3c504d698ac\": container with ID starting with 0fd6060ed3c8ce26aaa6f12ed87f756875627777eb4e718b6e90a3c504d698ac not found: ID does not exist" containerID="0fd6060ed3c8ce26aaa6f12ed87f756875627777eb4e718b6e90a3c504d698ac" Jul 28 17:35:42.692356 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.692325 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fd6060ed3c8ce26aaa6f12ed87f756875627777eb4e718b6e90a3c504d698ac"} err="failed to get container status \"0fd6060ed3c8ce26aaa6f12ed87f756875627777eb4e718b6e90a3c504d698ac\": rpc error: code = NotFound desc = could not find container \"0fd6060ed3c8ce26aaa6f12ed87f756875627777eb4e718b6e90a3c504d698ac\": container with ID starting with 0fd6060ed3c8ce26aaa6f12ed87f756875627777eb4e718b6e90a3c504d698ac not found: ID does not exist" Jul 28 17:35:42.700096 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.700075 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl"] Jul 28 17:35:42.712225 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:42.712201 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-949b0a76-openshift-default-77c6786d88-hl8gl"] Jul 28 17:35:44.120235 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:35:44.120204 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c614212-e197-44f3-84a3-e6f38a10aaef" path="/var/lib/kubelet/pods/2c614212-e197-44f3-84a3-e6f38a10aaef/volumes" Jul 28 17:36:53.875057 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:36:53.874979 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-133ebd2a-dlrlp"] Jul 28 17:36:53.875525 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:36:53.875437 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c614212-e197-44f3-84a3-e6f38a10aaef" containerName="istio-proxy" Jul 28 17:36:53.875525 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:36:53.875452 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c614212-e197-44f3-84a3-e6f38a10aaef" containerName="istio-proxy" Jul 28 17:36:53.875598 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:36:53.875536 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c614212-e197-44f3-84a3-e6f38a10aaef" containerName="istio-proxy" Jul 28 17:36:53.875598 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:36:53.875548 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c614212-e197-44f3-84a3-e6f38a10aaef" containerName="istio-proxy" Jul 28 17:36:53.878724 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:36:53.878703 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-133ebd2a-dlrlp" Jul 28 17:36:53.881458 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:36:53.881438 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tl4gw\"" Jul 28 17:36:53.894743 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:36:53.894707 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-133ebd2a-dlrlp"] Jul 28 17:36:53.990462 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:36:53.990427 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a652e983-c976-448d-8475-a50ff5c44f46-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-133ebd2a-dlrlp\" (UID: \"a652e983-c976-448d-8475-a50ff5c44f46\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-133ebd2a-dlrlp" Jul 28 17:36:54.091925 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:36:54.091892 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a652e983-c976-448d-8475-a50ff5c44f46-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-133ebd2a-dlrlp\" (UID: \"a652e983-c976-448d-8475-a50ff5c44f46\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-133ebd2a-dlrlp" Jul 28 17:36:54.092562 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:36:54.092541 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a652e983-c976-448d-8475-a50ff5c44f46-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-133ebd2a-dlrlp\" (UID: \"a652e983-c976-448d-8475-a50ff5c44f46\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-133ebd2a-dlrlp" Jul 28 17:36:54.199305 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:36:54.199214 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-133ebd2a-dlrlp" Jul 28 17:36:54.341781 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:36:54.341755 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-133ebd2a-dlrlp"] Jul 28 17:36:54.344404 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:36:54.344364 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda652e983_c976_448d_8475_a50ff5c44f46.slice/crio-29847afcb0dc9acf4488c9b61bcc80831af88552149d07996a8a47c95c53bb3d WatchSource:0}: Error finding container 29847afcb0dc9acf4488c9b61bcc80831af88552149d07996a8a47c95c53bb3d: Status 404 returned error can't find the container with id 29847afcb0dc9acf4488c9b61bcc80831af88552149d07996a8a47c95c53bb3d Jul 28 17:36:54.346763 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:36:54.346744 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 17:36:55.063099 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:36:55.063062 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-133ebd2a-dlrlp" event={"ID":"a652e983-c976-448d-8475-a50ff5c44f46","Type":"ContainerStarted","Data":"29847afcb0dc9acf4488c9b61bcc80831af88552149d07996a8a47c95c53bb3d"} Jul 28 17:36:57.076038 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:36:57.075995 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-133ebd2a-dlrlp" event={"ID":"a652e983-c976-448d-8475-a50ff5c44f46","Type":"ContainerStarted","Data":"9c6f28ce653120d308298dee2dc3b5863619f3b301454c83d2bc7fc1e4514524"} Jul 28 17:36:57.093838 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:36:57.093790 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-133ebd2a-dlrlp" podStartSLOduration=1.776151298 podStartE2EDuration="4.093774887s" totalImagesPullingTime="2.317623589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:36:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:36:57.092819343 +0000 UTC m=+1715.701750714" watchObservedRunningTime="2026-07-28 17:36:57.093774887 +0000 UTC m=+1715.702706257" Jul 28 17:36:58.082125 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:36:58.082076 2570 generic.go:356] "Generic (PLEG): container finished" podID="a652e983-c976-448d-8475-a50ff5c44f46" containerID="9c6f28ce653120d308298dee2dc3b5863619f3b301454c83d2bc7fc1e4514524" exitCode=0 Jul 28 17:36:58.082508 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:36:58.082150 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-133ebd2a-dlrlp" event={"ID":"a652e983-c976-448d-8475-a50ff5c44f46","Type":"ContainerDied","Data":"9c6f28ce653120d308298dee2dc3b5863619f3b301454c83d2bc7fc1e4514524"} Jul 28 17:36:59.222488 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:36:59.222461 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-133ebd2a-dlrlp" Jul 28 17:36:59.347910 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:36:59.347827 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a652e983-c976-448d-8475-a50ff5c44f46-maas-api-service-ca\") pod \"a652e983-c976-448d-8475-a50ff5c44f46\" (UID: \"a652e983-c976-448d-8475-a50ff5c44f46\") " Jul 28 17:36:59.348205 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:36:59.348184 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a652e983-c976-448d-8475-a50ff5c44f46-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a652e983-c976-448d-8475-a50ff5c44f46" (UID: "a652e983-c976-448d-8475-a50ff5c44f46"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:36:59.449367 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:36:59.449329 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a652e983-c976-448d-8475-a50ff5c44f46-maas-api-service-ca\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:37:00.093824 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:00.093795 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-133ebd2a-dlrlp" Jul 28 17:37:00.093996 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:00.093837 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-133ebd2a-dlrlp" event={"ID":"a652e983-c976-448d-8475-a50ff5c44f46","Type":"ContainerDied","Data":"29847afcb0dc9acf4488c9b61bcc80831af88552149d07996a8a47c95c53bb3d"} Jul 28 17:37:00.093996 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:00.093885 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="29847afcb0dc9acf4488c9b61bcc80831af88552149d07996a8a47c95c53bb3d" Jul 28 17:37:08.961531 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:08.961494 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-133ebd2a-dlrlp"] Jul 28 17:37:08.967557 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:08.967533 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-133ebd2a-dlrlp"] Jul 28 17:37:10.120236 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:10.120206 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a652e983-c976-448d-8475-a50ff5c44f46" path="/var/lib/kubelet/pods/a652e983-c976-448d-8475-a50ff5c44f46/volumes" Jul 28 17:37:10.998711 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:10.998676 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk"] Jul 28 17:37:10.999126 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:10.999101 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a652e983-c976-448d-8475-a50ff5c44f46" containerName="revoke-keys" Jul 28 17:37:10.999186 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:10.999129 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a652e983-c976-448d-8475-a50ff5c44f46" containerName="revoke-keys" Jul 28 17:37:10.999186 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:10.999153 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c614212-e197-44f3-84a3-e6f38a10aaef" containerName="istio-proxy" Jul 28 17:37:10.999186 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:10.999159 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c614212-e197-44f3-84a3-e6f38a10aaef" containerName="istio-proxy" Jul 28 17:37:10.999281 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:10.999228 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="a652e983-c976-448d-8475-a50ff5c44f46" containerName="revoke-keys" Jul 28 17:37:11.002283 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:11.002265 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:11.005664 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:11.005643 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-31d2617b-openshift-default-dockercfg-bll67\"" Jul 28 17:37:11.019844 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:11.019814 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk"] Jul 28 17:37:11.058213 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:11.058183 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c7b9b76-ade7-4432-9f2b-8b282671c17d-istio-data\") pod \"e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk\" (UID: \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\") " pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:11.058374 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:11.058230 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c7b9b76-ade7-4432-9f2b-8b282671c17d-workload-socket\") pod \"e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk\" (UID: \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\") " pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:11.058374 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:11.058253 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c7b9b76-ade7-4432-9f2b-8b282671c17d-istio-podinfo\") pod \"e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk\" (UID: \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\") " pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:11.058488 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:11.058371 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9d9q\" (UniqueName: \"kubernetes.io/projected/2c7b9b76-ade7-4432-9f2b-8b282671c17d-kube-api-access-x9d9q\") pod \"e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk\" (UID: \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\") " pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:11.058488 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:11.058405 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c7b9b76-ade7-4432-9f2b-8b282671c17d-istiod-ca-cert\") pod \"e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk\" (UID: \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\") " pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:11.058488 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:11.058442 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c7b9b76-ade7-4432-9f2b-8b282671c17d-istio-token\") pod \"e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk\" (UID: \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\") " pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:11.058488 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:11.058471 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c7b9b76-ade7-4432-9f2b-8b282671c17d-istio-envoy\") pod \"e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk\" (UID: \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\") " pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:11.058683 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:11.058501 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c7b9b76-ade7-4432-9f2b-8b282671c17d-credential-socket\") pod \"e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk\" (UID: \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\") " pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:11.058683 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:11.058639 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c7b9b76-ade7-4432-9f2b-8b282671c17d-workload-certs\") pod \"e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk\" (UID: \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\") " pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:11.160062 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:11.160029 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c7b9b76-ade7-4432-9f2b-8b282671c17d-workload-socket\") pod \"e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk\" (UID: \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\") " pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:11.160062 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:11.160066 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c7b9b76-ade7-4432-9f2b-8b282671c17d-istio-podinfo\") pod \"e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk\" (UID: \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\") " pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:11.160636 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:11.160216 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9d9q\" (UniqueName: \"kubernetes.io/projected/2c7b9b76-ade7-4432-9f2b-8b282671c17d-kube-api-access-x9d9q\") pod \"e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk\" (UID: \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\") " pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:11.160636 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:11.160253 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c7b9b76-ade7-4432-9f2b-8b282671c17d-istiod-ca-cert\") pod \"e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk\" (UID: \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\") " pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:11.160636 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:11.160300 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c7b9b76-ade7-4432-9f2b-8b282671c17d-istio-token\") pod \"e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk\" (UID: \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\") " pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:11.160636 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:11.160328 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c7b9b76-ade7-4432-9f2b-8b282671c17d-istio-envoy\") pod \"e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk\" (UID: \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\") " pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:11.160636 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:11.160356 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c7b9b76-ade7-4432-9f2b-8b282671c17d-credential-socket\") pod \"e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk\" (UID: \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\") " pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:11.160636 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:11.160430 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c7b9b76-ade7-4432-9f2b-8b282671c17d-workload-certs\") pod \"e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk\" (UID: \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\") " pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:11.160636 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:11.160459 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c7b9b76-ade7-4432-9f2b-8b282671c17d-istio-data\") pod \"e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk\" (UID: \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\") " pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:11.160636 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:11.160529 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c7b9b76-ade7-4432-9f2b-8b282671c17d-workload-socket\") pod \"e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk\" (UID: \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\") " pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:11.161061 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:11.160795 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c7b9b76-ade7-4432-9f2b-8b282671c17d-credential-socket\") pod \"e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk\" (UID: \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\") " pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:11.161061 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:11.160844 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c7b9b76-ade7-4432-9f2b-8b282671c17d-istio-data\") pod \"e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk\" (UID: \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\") " pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:11.161061 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:11.160926 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c7b9b76-ade7-4432-9f2b-8b282671c17d-workload-certs\") pod \"e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk\" (UID: \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\") " pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:11.161061 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:11.161013 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c7b9b76-ade7-4432-9f2b-8b282671c17d-istiod-ca-cert\") pod \"e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk\" (UID: \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\") " pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:11.163165 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:11.163104 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c7b9b76-ade7-4432-9f2b-8b282671c17d-istio-podinfo\") pod \"e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk\" (UID: \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\") " pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:11.163288 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:11.163252 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c7b9b76-ade7-4432-9f2b-8b282671c17d-istio-envoy\") pod \"e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk\" (UID: \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\") " pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:11.169662 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:11.169635 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c7b9b76-ade7-4432-9f2b-8b282671c17d-istio-token\") pod \"e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk\" (UID: \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\") " pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:11.169662 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:11.169659 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9d9q\" (UniqueName: \"kubernetes.io/projected/2c7b9b76-ade7-4432-9f2b-8b282671c17d-kube-api-access-x9d9q\") pod \"e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk\" (UID: \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\") " pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:11.312165 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:11.312067 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:11.462373 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:11.462346 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk"] Jul 28 17:37:11.463954 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:37:11.463926 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c7b9b76_ade7_4432_9f2b_8b282671c17d.slice/crio-30e158cdda13356959e551a12615a7137d76bd957d5cb72b25c5f8fe723b1b1e WatchSource:0}: Error finding container 30e158cdda13356959e551a12615a7137d76bd957d5cb72b25c5f8fe723b1b1e: Status 404 returned error can't find the container with id 30e158cdda13356959e551a12615a7137d76bd957d5cb72b25c5f8fe723b1b1e Jul 28 17:37:11.466461 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:11.466432 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 17:37:11.466520 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:11.466497 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 17:37:11.466554 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:11.466522 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 17:37:12.154350 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:12.154314 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" event={"ID":"2c7b9b76-ade7-4432-9f2b-8b282671c17d","Type":"ContainerStarted","Data":"47e876fec94d84be4cbde9966f23f062147eccf16d1960e8a46295c09c87b2a9"} Jul 28 17:37:12.154542 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:12.154376 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" event={"ID":"2c7b9b76-ade7-4432-9f2b-8b282671c17d","Type":"ContainerStarted","Data":"30e158cdda13356959e551a12615a7137d76bd957d5cb72b25c5f8fe723b1b1e"} Jul 28 17:37:12.186398 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:12.186302 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" podStartSLOduration=2.186283716 podStartE2EDuration="2.186283716s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:37:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:37:12.186027864 +0000 UTC m=+1730.794959234" watchObservedRunningTime="2026-07-28 17:37:12.186283716 +0000 UTC m=+1730.795215088" Jul 28 17:37:12.313029 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:12.312997 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:12.318215 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:12.318178 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:13.159448 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:13.159419 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:13.160555 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:13.160537 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:27.234190 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:27.234154 2570 generic.go:356] "Generic (PLEG): container finished" podID="2c7b9b76-ade7-4432-9f2b-8b282671c17d" containerID="47e876fec94d84be4cbde9966f23f062147eccf16d1960e8a46295c09c87b2a9" exitCode=0 Jul 28 17:37:27.234600 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:27.234204 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" event={"ID":"2c7b9b76-ade7-4432-9f2b-8b282671c17d","Type":"ContainerDied","Data":"47e876fec94d84be4cbde9966f23f062147eccf16d1960e8a46295c09c87b2a9"} Jul 28 17:37:27.234600 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:27.234566 2570 scope.go:117] "RemoveContainer" containerID="47e876fec94d84be4cbde9966f23f062147eccf16d1960e8a46295c09c87b2a9" Jul 28 17:37:27.235558 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:27.235528 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 17:37:27.235651 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:27.235613 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 17:37:27.235714 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:27.235660 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 17:37:28.160416 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:28.160384 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:28.245208 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:28.245162 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" event={"ID":"2c7b9b76-ade7-4432-9f2b-8b282671c17d","Type":"ContainerStarted","Data":"8cd5e8a4f452514c3b81fc2e6009599f7f6a4e4decfa387b5194f8882c92f64f"} Jul 28 17:37:28.312796 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:28.312754 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:28.314082 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:28.314056 2570 patch_prober.go:28] interesting pod/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 28 17:37:28.314216 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:28.314149 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" podUID="2c7b9b76-ade7-4432-9f2b-8b282671c17d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 28 17:37:29.313224 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:29.313188 2570 patch_prober.go:28] interesting pod/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 28 17:37:29.313625 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:29.313239 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" podUID="2c7b9b76-ade7-4432-9f2b-8b282671c17d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 28 17:37:30.313179 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:30.313141 2570 patch_prober.go:28] interesting pod/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 28 17:37:30.313367 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:30.313217 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" podUID="2c7b9b76-ade7-4432-9f2b-8b282671c17d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 28 17:37:31.312490 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:31.312454 2570 patch_prober.go:28] interesting pod/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 28 17:37:31.312660 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:31.312507 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" podUID="2c7b9b76-ade7-4432-9f2b-8b282671c17d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 28 17:37:32.312799 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:32.312764 2570 patch_prober.go:28] interesting pod/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 28 17:37:32.313214 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:32.312820 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" podUID="2c7b9b76-ade7-4432-9f2b-8b282671c17d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 28 17:37:33.317334 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:33.317302 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:33.317776 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:33.317594 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:33.318250 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:33.318233 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:53.722768 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:53.722732 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk"] Jul 28 17:37:53.723224 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:53.722932 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" podUID="2c7b9b76-ade7-4432-9f2b-8b282671c17d" containerName="istio-proxy" containerID="cri-o://8cd5e8a4f452514c3b81fc2e6009599f7f6a4e4decfa387b5194f8882c92f64f" gracePeriod=30 Jul 28 17:37:58.983499 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:58.983470 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:59.142981 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:59.142947 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x9d9q\" (UniqueName: \"kubernetes.io/projected/2c7b9b76-ade7-4432-9f2b-8b282671c17d-kube-api-access-x9d9q\") pod \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\" (UID: \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\") " Jul 28 17:37:59.143200 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:59.142992 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c7b9b76-ade7-4432-9f2b-8b282671c17d-istio-envoy\") pod \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\" (UID: \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\") " Jul 28 17:37:59.143200 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:59.143038 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c7b9b76-ade7-4432-9f2b-8b282671c17d-istio-data\") pod \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\" (UID: \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\") " Jul 28 17:37:59.143200 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:59.143059 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c7b9b76-ade7-4432-9f2b-8b282671c17d-istio-token\") pod \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\" (UID: \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\") " Jul 28 17:37:59.143200 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:59.143082 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c7b9b76-ade7-4432-9f2b-8b282671c17d-credential-socket\") pod \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\" (UID: \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\") " Jul 28 17:37:59.143200 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:59.143098 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c7b9b76-ade7-4432-9f2b-8b282671c17d-workload-socket\") pod \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\" (UID: \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\") " Jul 28 17:37:59.143200 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:59.143199 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c7b9b76-ade7-4432-9f2b-8b282671c17d-istiod-ca-cert\") pod \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\" (UID: \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\") " Jul 28 17:37:59.143531 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:59.143231 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c7b9b76-ade7-4432-9f2b-8b282671c17d-workload-certs\") pod \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\" (UID: \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\") " Jul 28 17:37:59.143531 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:59.143273 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c7b9b76-ade7-4432-9f2b-8b282671c17d-istio-podinfo\") pod \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\" (UID: \"2c7b9b76-ade7-4432-9f2b-8b282671c17d\") " Jul 28 17:37:59.143531 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:59.143419 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c7b9b76-ade7-4432-9f2b-8b282671c17d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2c7b9b76-ade7-4432-9f2b-8b282671c17d" (UID: "2c7b9b76-ade7-4432-9f2b-8b282671c17d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:37:59.143531 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:59.143490 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c7b9b76-ade7-4432-9f2b-8b282671c17d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2c7b9b76-ade7-4432-9f2b-8b282671c17d" (UID: "2c7b9b76-ade7-4432-9f2b-8b282671c17d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:37:59.143749 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:59.143626 2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c7b9b76-ade7-4432-9f2b-8b282671c17d-credential-socket\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:37:59.143749 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:59.143647 2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c7b9b76-ade7-4432-9f2b-8b282671c17d-workload-socket\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:37:59.143749 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:59.143657 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c7b9b76-ade7-4432-9f2b-8b282671c17d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2c7b9b76-ade7-4432-9f2b-8b282671c17d" (UID: "2c7b9b76-ade7-4432-9f2b-8b282671c17d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:37:59.143906 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:59.143778 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c7b9b76-ade7-4432-9f2b-8b282671c17d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2c7b9b76-ade7-4432-9f2b-8b282671c17d" (UID: "2c7b9b76-ade7-4432-9f2b-8b282671c17d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:37:59.143994 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:59.143969 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c7b9b76-ade7-4432-9f2b-8b282671c17d-istio-data" (OuterVolumeSpecName: "istio-data") pod "2c7b9b76-ade7-4432-9f2b-8b282671c17d" (UID: "2c7b9b76-ade7-4432-9f2b-8b282671c17d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:37:59.145672 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:59.145642 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c7b9b76-ade7-4432-9f2b-8b282671c17d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2c7b9b76-ade7-4432-9f2b-8b282671c17d" (UID: "2c7b9b76-ade7-4432-9f2b-8b282671c17d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:37:59.145672 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:59.145659 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c7b9b76-ade7-4432-9f2b-8b282671c17d-kube-api-access-x9d9q" (OuterVolumeSpecName: "kube-api-access-x9d9q") pod "2c7b9b76-ade7-4432-9f2b-8b282671c17d" (UID: "2c7b9b76-ade7-4432-9f2b-8b282671c17d"). InnerVolumeSpecName "kube-api-access-x9d9q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:37:59.145827 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:59.145769 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2c7b9b76-ade7-4432-9f2b-8b282671c17d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2c7b9b76-ade7-4432-9f2b-8b282671c17d" (UID: "2c7b9b76-ade7-4432-9f2b-8b282671c17d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 28 17:37:59.145889 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:59.145865 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c7b9b76-ade7-4432-9f2b-8b282671c17d-istio-token" (OuterVolumeSpecName: "istio-token") pod "2c7b9b76-ade7-4432-9f2b-8b282671c17d" (UID: "2c7b9b76-ade7-4432-9f2b-8b282671c17d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:37:59.245051 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:59.245013 2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c7b9b76-ade7-4432-9f2b-8b282671c17d-istiod-ca-cert\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:37:59.245051 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:59.245042 2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c7b9b76-ade7-4432-9f2b-8b282671c17d-workload-certs\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:37:59.245051 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:59.245051 2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c7b9b76-ade7-4432-9f2b-8b282671c17d-istio-podinfo\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:37:59.245051 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:59.245058 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9d9q\" (UniqueName: \"kubernetes.io/projected/2c7b9b76-ade7-4432-9f2b-8b282671c17d-kube-api-access-x9d9q\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:37:59.245326 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:59.245067 2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c7b9b76-ade7-4432-9f2b-8b282671c17d-istio-envoy\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:37:59.245326 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:59.245086 2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c7b9b76-ade7-4432-9f2b-8b282671c17d-istio-data\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:37:59.245326 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:59.245093 2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c7b9b76-ade7-4432-9f2b-8b282671c17d-istio-token\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:37:59.410765 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:59.410663 2570 generic.go:356] "Generic (PLEG): container finished" podID="2c7b9b76-ade7-4432-9f2b-8b282671c17d" containerID="8cd5e8a4f452514c3b81fc2e6009599f7f6a4e4decfa387b5194f8882c92f64f" exitCode=0 Jul 28 17:37:59.410945 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:59.410829 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" event={"ID":"2c7b9b76-ade7-4432-9f2b-8b282671c17d","Type":"ContainerDied","Data":"8cd5e8a4f452514c3b81fc2e6009599f7f6a4e4decfa387b5194f8882c92f64f"} Jul 28 17:37:59.410945 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:59.410879 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" event={"ID":"2c7b9b76-ade7-4432-9f2b-8b282671c17d","Type":"ContainerDied","Data":"30e158cdda13356959e551a12615a7137d76bd957d5cb72b25c5f8fe723b1b1e"} Jul 28 17:37:59.410945 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:59.410904 2570 scope.go:117] "RemoveContainer" containerID="8cd5e8a4f452514c3b81fc2e6009599f7f6a4e4decfa387b5194f8882c92f64f" Jul 28 17:37:59.411103 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:59.411057 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk" Jul 28 17:37:59.428171 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:59.428139 2570 scope.go:117] "RemoveContainer" containerID="47e876fec94d84be4cbde9966f23f062147eccf16d1960e8a46295c09c87b2a9" Jul 28 17:37:59.440101 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:59.440076 2570 scope.go:117] "RemoveContainer" containerID="8cd5e8a4f452514c3b81fc2e6009599f7f6a4e4decfa387b5194f8882c92f64f" Jul 28 17:37:59.440438 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:37:59.440415 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cd5e8a4f452514c3b81fc2e6009599f7f6a4e4decfa387b5194f8882c92f64f\": container with ID starting with 8cd5e8a4f452514c3b81fc2e6009599f7f6a4e4decfa387b5194f8882c92f64f not found: ID does not exist" containerID="8cd5e8a4f452514c3b81fc2e6009599f7f6a4e4decfa387b5194f8882c92f64f" Jul 28 17:37:59.440505 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:59.440448 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cd5e8a4f452514c3b81fc2e6009599f7f6a4e4decfa387b5194f8882c92f64f"} err="failed to get container status \"8cd5e8a4f452514c3b81fc2e6009599f7f6a4e4decfa387b5194f8882c92f64f\": rpc error: code = NotFound desc = could not find container \"8cd5e8a4f452514c3b81fc2e6009599f7f6a4e4decfa387b5194f8882c92f64f\": container with ID starting with 8cd5e8a4f452514c3b81fc2e6009599f7f6a4e4decfa387b5194f8882c92f64f not found: ID does not exist" Jul 28 17:37:59.440505 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:59.440467 2570 scope.go:117] "RemoveContainer" containerID="47e876fec94d84be4cbde9966f23f062147eccf16d1960e8a46295c09c87b2a9" Jul 28 17:37:59.440707 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:37:59.440687 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47e876fec94d84be4cbde9966f23f062147eccf16d1960e8a46295c09c87b2a9\": container with ID starting with 47e876fec94d84be4cbde9966f23f062147eccf16d1960e8a46295c09c87b2a9 not found: ID does not exist" containerID="47e876fec94d84be4cbde9966f23f062147eccf16d1960e8a46295c09c87b2a9" Jul 28 17:37:59.440768 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:59.440717 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47e876fec94d84be4cbde9966f23f062147eccf16d1960e8a46295c09c87b2a9"} err="failed to get container status \"47e876fec94d84be4cbde9966f23f062147eccf16d1960e8a46295c09c87b2a9\": rpc error: code = NotFound desc = could not find container \"47e876fec94d84be4cbde9966f23f062147eccf16d1960e8a46295c09c87b2a9\": container with ID starting with 47e876fec94d84be4cbde9966f23f062147eccf16d1960e8a46295c09c87b2a9 not found: ID does not exist" Jul 28 17:37:59.446817 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:59.446788 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk"] Jul 28 17:37:59.454374 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:37:59.454337 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-31d2617b-openshift-default-c8b7f7f5b-np6bk"] Jul 28 17:38:00.120563 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:38:00.120533 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c7b9b76-ade7-4432-9f2b-8b282671c17d" path="/var/lib/kubelet/pods/2c7b9b76-ade7-4432-9f2b-8b282671c17d/volumes" Jul 28 17:38:21.975730 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:38:21.975696 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-slfqf_ecb82800-8cf0-4ef9-926c-155172fad7ff/console-operator/1.log" Jul 28 17:38:21.978640 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:38:21.978620 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-slfqf_ecb82800-8cf0-4ef9-926c-155172fad7ff/console-operator/1.log" Jul 28 17:39:50.925661 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:39:50.925605 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d9be9a39-t4bgg"] Jul 28 17:39:50.926313 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:39:50.926296 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c7b9b76-ade7-4432-9f2b-8b282671c17d" containerName="istio-proxy" Jul 28 17:39:50.926365 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:39:50.926315 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c7b9b76-ade7-4432-9f2b-8b282671c17d" containerName="istio-proxy" Jul 28 17:39:50.926414 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:39:50.926405 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c7b9b76-ade7-4432-9f2b-8b282671c17d" containerName="istio-proxy" Jul 28 17:39:50.926450 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:39:50.926417 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c7b9b76-ade7-4432-9f2b-8b282671c17d" containerName="istio-proxy" Jul 28 17:39:50.929846 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:39:50.929828 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d9be9a39-t4bgg" Jul 28 17:39:50.932415 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:39:50.932389 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tl4gw\"" Jul 28 17:39:50.937013 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:39:50.936990 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d9be9a39-t4bgg"] Jul 28 17:39:51.000183 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:39:51.000151 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49ff0905-08b5-4fa5-b188-366c28922721-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d9be9a39-t4bgg\" (UID: \"49ff0905-08b5-4fa5-b188-366c28922721\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d9be9a39-t4bgg" Jul 28 17:39:51.100726 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:39:51.100695 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49ff0905-08b5-4fa5-b188-366c28922721-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d9be9a39-t4bgg\" (UID: \"49ff0905-08b5-4fa5-b188-366c28922721\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d9be9a39-t4bgg" Jul 28 17:39:51.101310 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:39:51.101292 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49ff0905-08b5-4fa5-b188-366c28922721-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d9be9a39-t4bgg\" (UID: \"49ff0905-08b5-4fa5-b188-366c28922721\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d9be9a39-t4bgg" Jul 28 17:39:51.243293 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:39:51.243208 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d9be9a39-t4bgg" Jul 28 17:39:51.387432 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:39:51.387403 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d9be9a39-t4bgg"] Jul 28 17:39:51.389598 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:39:51.389563 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49ff0905_08b5_4fa5_b188_366c28922721.slice/crio-da0737d332f83f369d859af2c2db1f375a3ac6bfc8932e61af0701d5433ec4ef WatchSource:0}: Error finding container da0737d332f83f369d859af2c2db1f375a3ac6bfc8932e61af0701d5433ec4ef: Status 404 returned error can't find the container with id da0737d332f83f369d859af2c2db1f375a3ac6bfc8932e61af0701d5433ec4ef Jul 28 17:39:52.012626 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:39:52.012592 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d9be9a39-t4bgg" event={"ID":"49ff0905-08b5-4fa5-b188-366c28922721","Type":"ContainerStarted","Data":"80cd09601bcbba5948e573e48262d186333b9930eb32f08bc9b2d5f39e30fe0e"} Jul 28 17:39:52.012997 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:39:52.012641 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d9be9a39-t4bgg" event={"ID":"49ff0905-08b5-4fa5-b188-366c28922721","Type":"ContainerStarted","Data":"da0737d332f83f369d859af2c2db1f375a3ac6bfc8932e61af0701d5433ec4ef"} Jul 28 17:39:52.030005 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:39:52.029961 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d9be9a39-t4bgg" podStartSLOduration=2.029947728 podStartE2EDuration="2.029947728s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:39:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:39:52.028865222 +0000 UTC m=+1890.637796591" watchObservedRunningTime="2026-07-28 17:39:52.029947728 +0000 UTC m=+1890.638879097" Jul 28 17:39:53.019683 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:39:53.019645 2570 generic.go:356] "Generic (PLEG): container finished" podID="49ff0905-08b5-4fa5-b188-366c28922721" containerID="80cd09601bcbba5948e573e48262d186333b9930eb32f08bc9b2d5f39e30fe0e" exitCode=0 Jul 28 17:39:53.020075 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:39:53.019709 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d9be9a39-t4bgg" event={"ID":"49ff0905-08b5-4fa5-b188-366c28922721","Type":"ContainerDied","Data":"80cd09601bcbba5948e573e48262d186333b9930eb32f08bc9b2d5f39e30fe0e"} Jul 28 17:39:54.171680 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:39:54.171656 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d9be9a39-t4bgg" Jul 28 17:39:54.229900 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:39:54.229868 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49ff0905-08b5-4fa5-b188-366c28922721-maas-api-service-ca\") pod \"49ff0905-08b5-4fa5-b188-366c28922721\" (UID: \"49ff0905-08b5-4fa5-b188-366c28922721\") " Jul 28 17:39:54.230274 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:39:54.230245 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49ff0905-08b5-4fa5-b188-366c28922721-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "49ff0905-08b5-4fa5-b188-366c28922721" (UID: "49ff0905-08b5-4fa5-b188-366c28922721"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:39:54.331197 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:39:54.331087 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49ff0905-08b5-4fa5-b188-366c28922721-maas-api-service-ca\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:39:55.031991 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:39:55.031963 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d9be9a39-t4bgg" Jul 28 17:39:55.032202 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:39:55.031993 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d9be9a39-t4bgg" event={"ID":"49ff0905-08b5-4fa5-b188-366c28922721","Type":"ContainerDied","Data":"da0737d332f83f369d859af2c2db1f375a3ac6bfc8932e61af0701d5433ec4ef"} Jul 28 17:39:55.032202 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:39:55.032039 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="da0737d332f83f369d859af2c2db1f375a3ac6bfc8932e61af0701d5433ec4ef" Jul 28 17:40:06.034497 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:40:06.034460 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d9be9a39-t4bgg"] Jul 28 17:40:06.036437 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:40:06.036413 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d9be9a39-t4bgg"] Jul 28 17:40:06.120081 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:40:06.120046 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49ff0905-08b5-4fa5-b188-366c28922721" path="/var/lib/kubelet/pods/49ff0905-08b5-4fa5-b188-366c28922721/volumes" Jul 28 17:43:22.017530 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:43:22.017498 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-slfqf_ecb82800-8cf0-4ef9-926c-155172fad7ff/console-operator/1.log" Jul 28 17:43:22.020320 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:43:22.020302 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-slfqf_ecb82800-8cf0-4ef9-926c-155172fad7ff/console-operator/1.log" Jul 28 17:43:22.339255 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:43:22.339178 2570 scope.go:117] "RemoveContainer" containerID="9c6f28ce653120d308298dee2dc3b5863619f3b301454c83d2bc7fc1e4514524" Jul 28 17:45:00.152282 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:00.152238 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754345-8pgwg"] Jul 28 17:45:00.152819 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:00.152650 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49ff0905-08b5-4fa5-b188-366c28922721" containerName="revoke-keys" Jul 28 17:45:00.152819 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:00.152664 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="49ff0905-08b5-4fa5-b188-366c28922721" containerName="revoke-keys" Jul 28 17:45:00.152819 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:00.152684 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c7b9b76-ade7-4432-9f2b-8b282671c17d" containerName="istio-proxy" Jul 28 17:45:00.152819 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:00.152690 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c7b9b76-ade7-4432-9f2b-8b282671c17d" containerName="istio-proxy" Jul 28 17:45:00.152819 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:00.152777 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="49ff0905-08b5-4fa5-b188-366c28922721" containerName="revoke-keys" Jul 28 17:45:00.156085 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:00.156066 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8796a3-29754345-9xmg7"] Jul 28 17:45:00.156232 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:00.156196 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754345-8pgwg" Jul 28 17:45:00.159466 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:00.159449 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7dbdbf-29754345-rsbtm"] Jul 28 17:45:00.159567 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:00.159484 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tl4gw\"" Jul 28 17:45:00.159618 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:00.159564 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8796a3-29754345-9xmg7" Jul 28 17:45:00.162817 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:00.162797 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7dbdbf-29754345-rsbtm" Jul 28 17:45:00.166732 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:00.166711 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7dbdbf-29754345-rsbtm"] Jul 28 17:45:00.169233 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:00.169214 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754345-8pgwg" Jul 28 17:45:00.169704 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:00.169681 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8796a3-29754345-9xmg7"] Jul 28 17:45:00.173022 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:00.172417 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754345-8pgwg"] Jul 28 17:45:00.180853 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:00.180834 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8796a3-29754345-9xmg7" Jul 28 17:45:00.225532 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:00.225510 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7dbdbf-29754345-rsbtm" Jul 28 17:45:00.388863 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:00.388830 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8796a3-29754345-9xmg7"] Jul 28 17:45:00.390890 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:45:00.390859 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2b655ca_f594_4b60_9b0a_02f984e3e970.slice/crio-422826dd17f05a310f12c90f24665e8a4fa9738e21684c60e76cba28167ff3d5 WatchSource:0}: Error finding container 422826dd17f05a310f12c90f24665e8a4fa9738e21684c60e76cba28167ff3d5: Status 404 returned error can't find the container with id 422826dd17f05a310f12c90f24665e8a4fa9738e21684c60e76cba28167ff3d5 Jul 28 17:45:00.419312 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:00.419290 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7dbdbf-29754345-rsbtm"] Jul 28 17:45:00.422229 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:45:00.422204 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod016a4ee4_bdfc_46d9_a626_c69f7fc59685.slice/crio-1784f8705dcfc0c516294003fbaddd4b87dd43c39eba33504bccfcf203d17c03 WatchSource:0}: Error finding container 1784f8705dcfc0c516294003fbaddd4b87dd43c39eba33504bccfcf203d17c03: Status 404 returned error can't find the container with id 1784f8705dcfc0c516294003fbaddd4b87dd43c39eba33504bccfcf203d17c03 Jul 28 17:45:00.550526 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:00.550494 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754345-8pgwg"] Jul 28 17:45:00.551507 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:45:00.551479 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd167ac2_c609_4a60_9bc1_4c26d25de67c.slice/crio-cf28e59b3f6b0ec00fe607a3a2ee80665b85a861d75fe3a5320cbe7a8b7c6c4a WatchSource:0}: Error finding container cf28e59b3f6b0ec00fe607a3a2ee80665b85a861d75fe3a5320cbe7a8b7c6c4a: Status 404 returned error can't find the container with id cf28e59b3f6b0ec00fe607a3a2ee80665b85a861d75fe3a5320cbe7a8b7c6c4a Jul 28 17:45:00.648524 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:00.648499 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8796a3-29754345-9xmg7" event={"ID":"a2b655ca-f594-4b60-9b0a-02f984e3e970","Type":"ContainerStarted","Data":"422826dd17f05a310f12c90f24665e8a4fa9738e21684c60e76cba28167ff3d5"} Jul 28 17:45:00.649669 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:00.649647 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7dbdbf-29754345-rsbtm" event={"ID":"016a4ee4-bdfc-46d9-a626-c69f7fc59685","Type":"ContainerStarted","Data":"1784f8705dcfc0c516294003fbaddd4b87dd43c39eba33504bccfcf203d17c03"} Jul 28 17:45:00.650802 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:00.650782 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754345-8pgwg" event={"ID":"dd167ac2-c609-4a60-9bc1-4c26d25de67c","Type":"ContainerStarted","Data":"cf28e59b3f6b0ec00fe607a3a2ee80665b85a861d75fe3a5320cbe7a8b7c6c4a"} Jul 28 17:45:01.661091 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:01.661055 2570 generic.go:356] "Generic (PLEG): container finished" podID="a2b655ca-f594-4b60-9b0a-02f984e3e970" containerID="8a906c2f0eb18b7dd9a74dae5e27d396f70037702b44f543682c631f374ef7b9" exitCode=0 Jul 28 17:45:01.661514 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:01.661135 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8796a3-29754345-9xmg7" event={"ID":"a2b655ca-f594-4b60-9b0a-02f984e3e970","Type":"ContainerDied","Data":"8a906c2f0eb18b7dd9a74dae5e27d396f70037702b44f543682c631f374ef7b9"} Jul 28 17:45:01.662549 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:01.662527 2570 generic.go:356] "Generic (PLEG): container finished" podID="016a4ee4-bdfc-46d9-a626-c69f7fc59685" containerID="4273b3c3c450dfd35d6fbf191d9eb55f0311792c7ac92adfffaba5e30be94e88" exitCode=0 Jul 28 17:45:01.662662 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:01.662596 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7dbdbf-29754345-rsbtm" event={"ID":"016a4ee4-bdfc-46d9-a626-c69f7fc59685","Type":"ContainerDied","Data":"4273b3c3c450dfd35d6fbf191d9eb55f0311792c7ac92adfffaba5e30be94e88"} Jul 28 17:45:01.663929 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:01.663911 2570 generic.go:356] "Generic (PLEG): container finished" podID="dd167ac2-c609-4a60-9bc1-4c26d25de67c" containerID="90c10e96b46b01dcec0ceb0aacfd0704fbd398d748eeb438d96cdcdeac580e28" exitCode=0 Jul 28 17:45:01.663998 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:01.663970 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754345-8pgwg" event={"ID":"dd167ac2-c609-4a60-9bc1-4c26d25de67c","Type":"ContainerDied","Data":"90c10e96b46b01dcec0ceb0aacfd0704fbd398d748eeb438d96cdcdeac580e28"} Jul 28 17:45:02.827908 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:02.827871 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754345-8pgwg" Jul 28 17:45:02.849388 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:02.849136 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7dbdbf-29754345-rsbtm" Jul 28 17:45:02.866333 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:02.866292 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8796a3-29754345-9xmg7" Jul 28 17:45:03.676001 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:03.675972 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754345-8pgwg" Jul 28 17:45:03.676237 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:03.676011 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754345-8pgwg" event={"ID":"dd167ac2-c609-4a60-9bc1-4c26d25de67c","Type":"ContainerDied","Data":"cf28e59b3f6b0ec00fe607a3a2ee80665b85a861d75fe3a5320cbe7a8b7c6c4a"} Jul 28 17:45:03.676237 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:03.676061 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cf28e59b3f6b0ec00fe607a3a2ee80665b85a861d75fe3a5320cbe7a8b7c6c4a" Jul 28 17:45:03.677413 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:03.677386 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8796a3-29754345-9xmg7" Jul 28 17:45:03.677555 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:03.677423 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8796a3-29754345-9xmg7" event={"ID":"a2b655ca-f594-4b60-9b0a-02f984e3e970","Type":"ContainerDied","Data":"422826dd17f05a310f12c90f24665e8a4fa9738e21684c60e76cba28167ff3d5"} Jul 28 17:45:03.677555 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:03.677456 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="422826dd17f05a310f12c90f24665e8a4fa9738e21684c60e76cba28167ff3d5" Jul 28 17:45:03.678677 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:03.678656 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7dbdbf-29754345-rsbtm" event={"ID":"016a4ee4-bdfc-46d9-a626-c69f7fc59685","Type":"ContainerDied","Data":"1784f8705dcfc0c516294003fbaddd4b87dd43c39eba33504bccfcf203d17c03"} Jul 28 17:45:03.678775 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:03.678685 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7dbdbf-29754345-rsbtm" Jul 28 17:45:03.678775 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:03.678695 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1784f8705dcfc0c516294003fbaddd4b87dd43c39eba33504bccfcf203d17c03" Jul 28 17:45:08.991195 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:08.991161 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7dbdbf-ztltr"] Jul 28 17:45:08.991578 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:08.991569 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2b655ca-f594-4b60-9b0a-02f984e3e970" containerName="cleanup" Jul 28 17:45:08.991631 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:08.991581 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2b655ca-f594-4b60-9b0a-02f984e3e970" containerName="cleanup" Jul 28 17:45:08.991631 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:08.991607 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="016a4ee4-bdfc-46d9-a626-c69f7fc59685" containerName="cleanup" Jul 28 17:45:08.991631 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:08.991613 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="016a4ee4-bdfc-46d9-a626-c69f7fc59685" containerName="cleanup" Jul 28 17:45:08.991631 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:08.991622 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd167ac2-c609-4a60-9bc1-4c26d25de67c" containerName="cleanup" Jul 28 17:45:08.991631 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:08.991628 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd167ac2-c609-4a60-9bc1-4c26d25de67c" containerName="cleanup" Jul 28 17:45:08.991787 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:08.991685 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd167ac2-c609-4a60-9bc1-4c26d25de67c" containerName="cleanup" Jul 28 17:45:08.991787 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:08.991695 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2b655ca-f594-4b60-9b0a-02f984e3e970" containerName="cleanup" Jul 28 17:45:08.991787 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:08.991701 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="016a4ee4-bdfc-46d9-a626-c69f7fc59685" containerName="cleanup" Jul 28 17:45:08.994884 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:08.994866 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7dbdbf-ztltr" Jul 28 17:45:08.997644 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:08.997619 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tl4gw\"" Jul 28 17:45:09.003050 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:09.003026 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7dbdbf-ztltr"] Jul 28 17:45:09.102872 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:09.102837 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12456e75-47e4-4c4c-a1bd-567f394228f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-7dbdbf-ztltr\" (UID: \"12456e75-47e4-4c4c-a1bd-567f394228f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7dbdbf-ztltr" Jul 28 17:45:09.204216 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:09.204179 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12456e75-47e4-4c4c-a1bd-567f394228f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-7dbdbf-ztltr\" (UID: \"12456e75-47e4-4c4c-a1bd-567f394228f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7dbdbf-ztltr" Jul 28 17:45:09.204810 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:09.204787 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12456e75-47e4-4c4c-a1bd-567f394228f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-7dbdbf-ztltr\" (UID: \"12456e75-47e4-4c4c-a1bd-567f394228f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7dbdbf-ztltr" Jul 28 17:45:09.308713 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:09.308620 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7dbdbf-ztltr" Jul 28 17:45:09.444813 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:09.444787 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7dbdbf-ztltr"] Jul 28 17:45:09.447126 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:45:09.447088 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12456e75_47e4_4c4c_a1bd_567f394228f6.slice/crio-209810d135024b4ca4959709e7ccd403988a58e8b799881b64b87e98216fc88b WatchSource:0}: Error finding container 209810d135024b4ca4959709e7ccd403988a58e8b799881b64b87e98216fc88b: Status 404 returned error can't find the container with id 209810d135024b4ca4959709e7ccd403988a58e8b799881b64b87e98216fc88b Jul 28 17:45:09.714879 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:09.714842 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7dbdbf-ztltr" event={"ID":"12456e75-47e4-4c4c-a1bd-567f394228f6","Type":"ContainerStarted","Data":"209810d135024b4ca4959709e7ccd403988a58e8b799881b64b87e98216fc88b"} Jul 28 17:45:10.721815 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:10.721782 2570 generic.go:356] "Generic (PLEG): container finished" podID="12456e75-47e4-4c4c-a1bd-567f394228f6" containerID="7a4ba09b9f5a89f60fd7d13541d59d3fdea7820e49c97bb65efcb1cb8f237033" exitCode=0 Jul 28 17:45:10.722247 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:10.721881 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7dbdbf-ztltr" event={"ID":"12456e75-47e4-4c4c-a1bd-567f394228f6","Type":"ContainerDied","Data":"7a4ba09b9f5a89f60fd7d13541d59d3fdea7820e49c97bb65efcb1cb8f237033"} Jul 28 17:45:11.877601 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:11.877577 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7dbdbf-ztltr" Jul 28 17:45:12.032926 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:12.032818 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12456e75-47e4-4c4c-a1bd-567f394228f6-maas-api-service-ca\") pod \"12456e75-47e4-4c4c-a1bd-567f394228f6\" (UID: \"12456e75-47e4-4c4c-a1bd-567f394228f6\") " Jul 28 17:45:12.033227 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:12.033202 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12456e75-47e4-4c4c-a1bd-567f394228f6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "12456e75-47e4-4c4c-a1bd-567f394228f6" (UID: "12456e75-47e4-4c4c-a1bd-567f394228f6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:45:12.134580 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:12.134549 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12456e75-47e4-4c4c-a1bd-567f394228f6-maas-api-service-ca\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:45:12.733563 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:12.733522 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7dbdbf-ztltr" event={"ID":"12456e75-47e4-4c4c-a1bd-567f394228f6","Type":"ContainerDied","Data":"209810d135024b4ca4959709e7ccd403988a58e8b799881b64b87e98216fc88b"} Jul 28 17:45:12.733563 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:12.733547 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7dbdbf-ztltr" Jul 28 17:45:12.733812 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:12.733584 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="209810d135024b4ca4959709e7ccd403988a58e8b799881b64b87e98216fc88b" Jul 28 17:45:19.494126 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:19.494086 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7dbdbf-29754345-rsbtm"] Jul 28 17:45:19.496810 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:19.496786 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7dbdbf-29754345-rsbtm"] Jul 28 17:45:20.120470 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:20.120435 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="016a4ee4-bdfc-46d9-a626-c69f7fc59685" path="/var/lib/kubelet/pods/016a4ee4-bdfc-46d9-a626-c69f7fc59685/volumes" Jul 28 17:45:24.080571 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:24.080531 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7dbdbf-ztltr"] Jul 28 17:45:24.082933 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:24.082911 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7dbdbf-ztltr"] Jul 28 17:45:24.120407 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:24.120382 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12456e75-47e4-4c4c-a1bd-567f394228f6" path="/var/lib/kubelet/pods/12456e75-47e4-4c4c-a1bd-567f394228f6/volumes" Jul 28 17:45:42.395204 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:42.395165 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8796a3-29754345-9xmg7"] Jul 28 17:45:42.397947 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:42.397923 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8796a3-29754345-9xmg7"] Jul 28 17:45:44.120454 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:45:44.120416 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2b655ca-f594-4b60-9b0a-02f984e3e970" path="/var/lib/kubelet/pods/a2b655ca-f594-4b60-9b0a-02f984e3e970/volumes" Jul 28 17:46:18.328967 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:18.328926 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7869f9-4mt69"] Jul 28 17:46:18.329589 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:18.329569 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12456e75-47e4-4c4c-a1bd-567f394228f6" containerName="revoke-keys" Jul 28 17:46:18.329635 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:18.329592 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="12456e75-47e4-4c4c-a1bd-567f394228f6" containerName="revoke-keys" Jul 28 17:46:18.329708 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:18.329697 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="12456e75-47e4-4c4c-a1bd-567f394228f6" containerName="revoke-keys" Jul 28 17:46:18.333361 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:18.333336 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7869f9-4mt69" Jul 28 17:46:18.336402 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:18.336379 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tl4gw\"" Jul 28 17:46:18.341446 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:18.341425 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7869f9-4mt69"] Jul 28 17:46:18.376725 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:18.376695 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0066bc88-1944-424e-b77d-d2e415190e50-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-7869f9-4mt69\" (UID: \"0066bc88-1944-424e-b77d-d2e415190e50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7869f9-4mt69" Jul 28 17:46:18.478132 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:18.478082 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0066bc88-1944-424e-b77d-d2e415190e50-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-7869f9-4mt69\" (UID: \"0066bc88-1944-424e-b77d-d2e415190e50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7869f9-4mt69" Jul 28 17:46:18.478687 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:18.478667 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0066bc88-1944-424e-b77d-d2e415190e50-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-7869f9-4mt69\" (UID: \"0066bc88-1944-424e-b77d-d2e415190e50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7869f9-4mt69" Jul 28 17:46:18.645943 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:18.645904 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7869f9-4mt69" Jul 28 17:46:18.795536 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:18.795424 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7869f9-4mt69"] Jul 28 17:46:18.798183 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:46:18.798157 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0066bc88_1944_424e_b77d_d2e415190e50.slice/crio-94d2233d252cdd0961ddf816d4fcedac9dd3880102668ccd39b1e30ea700c2d2 WatchSource:0}: Error finding container 94d2233d252cdd0961ddf816d4fcedac9dd3880102668ccd39b1e30ea700c2d2: Status 404 returned error can't find the container with id 94d2233d252cdd0961ddf816d4fcedac9dd3880102668ccd39b1e30ea700c2d2 Jul 28 17:46:19.086801 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:19.086762 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7869f9-4mt69" event={"ID":"0066bc88-1944-424e-b77d-d2e415190e50","Type":"ContainerStarted","Data":"94d2233d252cdd0961ddf816d4fcedac9dd3880102668ccd39b1e30ea700c2d2"} Jul 28 17:46:20.093163 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:20.093127 2570 generic.go:356] "Generic (PLEG): container finished" podID="0066bc88-1944-424e-b77d-d2e415190e50" containerID="1a4df7b12672f7369e60ab3910a09d17a8519691f47db4ea09a79578856f113a" exitCode=0 Jul 28 17:46:20.093564 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:20.093207 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7869f9-4mt69" event={"ID":"0066bc88-1944-424e-b77d-d2e415190e50","Type":"ContainerDied","Data":"1a4df7b12672f7369e60ab3910a09d17a8519691f47db4ea09a79578856f113a"} Jul 28 17:46:21.240147 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:21.240096 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7869f9-4mt69" Jul 28 17:46:21.308853 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:21.308818 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0066bc88-1944-424e-b77d-d2e415190e50-maas-api-service-ca\") pod \"0066bc88-1944-424e-b77d-d2e415190e50\" (UID: \"0066bc88-1944-424e-b77d-d2e415190e50\") " Jul 28 17:46:21.309208 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:21.309183 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0066bc88-1944-424e-b77d-d2e415190e50-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0066bc88-1944-424e-b77d-d2e415190e50" (UID: "0066bc88-1944-424e-b77d-d2e415190e50"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:46:21.409634 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:21.409601 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0066bc88-1944-424e-b77d-d2e415190e50-maas-api-service-ca\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:46:22.104403 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:22.104367 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7869f9-4mt69" Jul 28 17:46:22.104576 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:22.104398 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7869f9-4mt69" event={"ID":"0066bc88-1944-424e-b77d-d2e415190e50","Type":"ContainerDied","Data":"94d2233d252cdd0961ddf816d4fcedac9dd3880102668ccd39b1e30ea700c2d2"} Jul 28 17:46:22.104576 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:22.104455 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="94d2233d252cdd0961ddf816d4fcedac9dd3880102668ccd39b1e30ea700c2d2" Jul 28 17:46:22.377925 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:22.377900 2570 scope.go:117] "RemoveContainer" containerID="80cd09601bcbba5948e573e48262d186333b9930eb32f08bc9b2d5f39e30fe0e" Jul 28 17:46:38.430718 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:38.430680 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7869f9-4mt69"] Jul 28 17:46:38.434764 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:38.434736 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7869f9-4mt69"] Jul 28 17:46:40.120540 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:40.120502 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0066bc88-1944-424e-b77d-d2e415190e50" path="/var/lib/kubelet/pods/0066bc88-1944-424e-b77d-d2e415190e50/volumes" Jul 28 17:46:46.248444 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:46.248403 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-29d9e9-t8fzk"] Jul 28 17:46:46.249005 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:46.248984 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0066bc88-1944-424e-b77d-d2e415190e50" containerName="revoke-keys" Jul 28 17:46:46.249096 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:46.249007 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="0066bc88-1944-424e-b77d-d2e415190e50" containerName="revoke-keys" Jul 28 17:46:46.249169 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:46.249107 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="0066bc88-1944-424e-b77d-d2e415190e50" containerName="revoke-keys" Jul 28 17:46:46.252398 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:46.252378 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-29d9e9-t8fzk" Jul 28 17:46:46.255486 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:46.255465 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tl4gw\"" Jul 28 17:46:46.266666 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:46.266639 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-29d9e9-t8fzk"] Jul 28 17:46:46.353142 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:46.353078 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/467748c8-bd4a-4002-a2f6-71125c808747-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-29d9e9-t8fzk\" (UID: \"467748c8-bd4a-4002-a2f6-71125c808747\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-29d9e9-t8fzk" Jul 28 17:46:46.454521 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:46.454482 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/467748c8-bd4a-4002-a2f6-71125c808747-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-29d9e9-t8fzk\" (UID: \"467748c8-bd4a-4002-a2f6-71125c808747\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-29d9e9-t8fzk" Jul 28 17:46:46.455107 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:46.455087 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/467748c8-bd4a-4002-a2f6-71125c808747-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-29d9e9-t8fzk\" (UID: \"467748c8-bd4a-4002-a2f6-71125c808747\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-29d9e9-t8fzk" Jul 28 17:46:46.563076 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:46.562976 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-29d9e9-t8fzk" Jul 28 17:46:46.710631 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:46.710602 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-29d9e9-t8fzk"] Jul 28 17:46:46.712697 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:46:46.712664 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod467748c8_bd4a_4002_a2f6_71125c808747.slice/crio-6d53d1cbf1d8b7d3f5bdb77bee484d6c174e432722f6d7e377f062c1545fffd4 WatchSource:0}: Error finding container 6d53d1cbf1d8b7d3f5bdb77bee484d6c174e432722f6d7e377f062c1545fffd4: Status 404 returned error can't find the container with id 6d53d1cbf1d8b7d3f5bdb77bee484d6c174e432722f6d7e377f062c1545fffd4 Jul 28 17:46:47.241097 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:47.241056 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-29d9e9-t8fzk" event={"ID":"467748c8-bd4a-4002-a2f6-71125c808747","Type":"ContainerStarted","Data":"daa3f53e3308fd2d193a4e61b82dcf95a1f473cf0b0b10ac0902826d1bb8301e"} Jul 28 17:46:47.241279 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:47.241146 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-29d9e9-t8fzk" event={"ID":"467748c8-bd4a-4002-a2f6-71125c808747","Type":"ContainerStarted","Data":"6d53d1cbf1d8b7d3f5bdb77bee484d6c174e432722f6d7e377f062c1545fffd4"} Jul 28 17:46:47.281375 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:47.281324 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-29d9e9-t8fzk" podStartSLOduration=1.281308735 podStartE2EDuration="1.281308735s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:46:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:46:47.279996035 +0000 UTC m=+2305.888927406" watchObservedRunningTime="2026-07-28 17:46:47.281308735 +0000 UTC m=+2305.890240104" Jul 28 17:46:48.248759 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:48.248725 2570 generic.go:356] "Generic (PLEG): container finished" podID="467748c8-bd4a-4002-a2f6-71125c808747" containerID="daa3f53e3308fd2d193a4e61b82dcf95a1f473cf0b0b10ac0902826d1bb8301e" exitCode=0 Jul 28 17:46:48.248961 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:48.248806 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-29d9e9-t8fzk" event={"ID":"467748c8-bd4a-4002-a2f6-71125c808747","Type":"ContainerDied","Data":"daa3f53e3308fd2d193a4e61b82dcf95a1f473cf0b0b10ac0902826d1bb8301e"} Jul 28 17:46:49.399368 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:49.399341 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-29d9e9-t8fzk" Jul 28 17:46:49.483054 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:49.483024 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/467748c8-bd4a-4002-a2f6-71125c808747-maas-api-service-ca\") pod \"467748c8-bd4a-4002-a2f6-71125c808747\" (UID: \"467748c8-bd4a-4002-a2f6-71125c808747\") " Jul 28 17:46:49.483399 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:49.483378 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/467748c8-bd4a-4002-a2f6-71125c808747-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "467748c8-bd4a-4002-a2f6-71125c808747" (UID: "467748c8-bd4a-4002-a2f6-71125c808747"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:46:49.584301 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:49.584210 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/467748c8-bd4a-4002-a2f6-71125c808747-maas-api-service-ca\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:46:50.261172 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:50.261141 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-29d9e9-t8fzk" Jul 28 17:46:50.261339 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:50.261170 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-29d9e9-t8fzk" event={"ID":"467748c8-bd4a-4002-a2f6-71125c808747","Type":"ContainerDied","Data":"6d53d1cbf1d8b7d3f5bdb77bee484d6c174e432722f6d7e377f062c1545fffd4"} Jul 28 17:46:50.261339 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:46:50.261217 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6d53d1cbf1d8b7d3f5bdb77bee484d6c174e432722f6d7e377f062c1545fffd4" Jul 28 17:47:06.375582 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:06.375549 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-29d9e9-t8fzk"] Jul 28 17:47:06.384661 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:06.384634 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-29d9e9-t8fzk"] Jul 28 17:47:08.120020 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:08.119988 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="467748c8-bd4a-4002-a2f6-71125c808747" path="/var/lib/kubelet/pods/467748c8-bd4a-4002-a2f6-71125c808747/volumes" Jul 28 17:47:23.156067 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:23.156034 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx"] Jul 28 17:47:23.156563 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:23.156471 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="467748c8-bd4a-4002-a2f6-71125c808747" containerName="revoke-keys" Jul 28 17:47:23.156563 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:23.156484 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="467748c8-bd4a-4002-a2f6-71125c808747" containerName="revoke-keys" Jul 28 17:47:23.156563 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:23.156562 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="467748c8-bd4a-4002-a2f6-71125c808747" containerName="revoke-keys" Jul 28 17:47:23.160026 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:23.160007 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" Jul 28 17:47:23.163730 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:23.163707 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-f24b7c-openshift-default-dockercfg-c4mxv\"" Jul 28 17:47:23.176751 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:23.176726 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx"] Jul 28 17:47:23.302571 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:23.302529 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38532552-35c9-4e91-a557-9a176b2d7f22-istio-token\") pod \"e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx\" (UID: \"38532552-35c9-4e91-a557-9a176b2d7f22\") " pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" Jul 28 17:47:23.302752 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:23.302581 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38532552-35c9-4e91-a557-9a176b2d7f22-istiod-ca-cert\") pod \"e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx\" (UID: \"38532552-35c9-4e91-a557-9a176b2d7f22\") " pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" Jul 28 17:47:23.302752 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:23.302604 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38532552-35c9-4e91-a557-9a176b2d7f22-istio-envoy\") pod \"e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx\" (UID: \"38532552-35c9-4e91-a557-9a176b2d7f22\") " pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" Jul 28 17:47:23.302752 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:23.302629 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38532552-35c9-4e91-a557-9a176b2d7f22-workload-socket\") pod \"e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx\" (UID: \"38532552-35c9-4e91-a557-9a176b2d7f22\") " pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" Jul 28 17:47:23.302897 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:23.302765 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mn5hw\" (UniqueName: \"kubernetes.io/projected/38532552-35c9-4e91-a557-9a176b2d7f22-kube-api-access-mn5hw\") pod \"e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx\" (UID: \"38532552-35c9-4e91-a557-9a176b2d7f22\") " pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" Jul 28 17:47:23.302897 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:23.302804 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38532552-35c9-4e91-a557-9a176b2d7f22-workload-certs\") pod \"e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx\" (UID: \"38532552-35c9-4e91-a557-9a176b2d7f22\") " pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" Jul 28 17:47:23.302897 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:23.302827 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38532552-35c9-4e91-a557-9a176b2d7f22-istio-data\") pod \"e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx\" (UID: \"38532552-35c9-4e91-a557-9a176b2d7f22\") " pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" Jul 28 17:47:23.302897 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:23.302844 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38532552-35c9-4e91-a557-9a176b2d7f22-istio-podinfo\") pod \"e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx\" (UID: \"38532552-35c9-4e91-a557-9a176b2d7f22\") " pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" Jul 28 17:47:23.302897 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:23.302887 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38532552-35c9-4e91-a557-9a176b2d7f22-credential-socket\") pod \"e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx\" (UID: \"38532552-35c9-4e91-a557-9a176b2d7f22\") " pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" Jul 28 17:47:23.403842 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:23.403801 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mn5hw\" (UniqueName: \"kubernetes.io/projected/38532552-35c9-4e91-a557-9a176b2d7f22-kube-api-access-mn5hw\") pod \"e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx\" (UID: \"38532552-35c9-4e91-a557-9a176b2d7f22\") " pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" Jul 28 17:47:23.404049 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:23.403856 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38532552-35c9-4e91-a557-9a176b2d7f22-workload-certs\") pod \"e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx\" (UID: \"38532552-35c9-4e91-a557-9a176b2d7f22\") " pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" Jul 28 17:47:23.404049 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:23.403885 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38532552-35c9-4e91-a557-9a176b2d7f22-istio-data\") pod \"e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx\" (UID: \"38532552-35c9-4e91-a557-9a176b2d7f22\") " pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" Jul 28 17:47:23.404049 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:23.403909 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38532552-35c9-4e91-a557-9a176b2d7f22-istio-podinfo\") pod \"e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx\" (UID: \"38532552-35c9-4e91-a557-9a176b2d7f22\") " pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" Jul 28 17:47:23.404049 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:23.403941 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38532552-35c9-4e91-a557-9a176b2d7f22-credential-socket\") pod \"e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx\" (UID: \"38532552-35c9-4e91-a557-9a176b2d7f22\") " pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" Jul 28 17:47:23.404049 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:23.403990 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38532552-35c9-4e91-a557-9a176b2d7f22-istio-token\") pod \"e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx\" (UID: \"38532552-35c9-4e91-a557-9a176b2d7f22\") " pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" Jul 28 17:47:23.404049 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:23.404027 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38532552-35c9-4e91-a557-9a176b2d7f22-istiod-ca-cert\") pod \"e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx\" (UID: \"38532552-35c9-4e91-a557-9a176b2d7f22\") " pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" Jul 28 17:47:23.404421 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:23.404055 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38532552-35c9-4e91-a557-9a176b2d7f22-istio-envoy\") pod \"e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx\" (UID: \"38532552-35c9-4e91-a557-9a176b2d7f22\") " pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" Jul 28 17:47:23.404421 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:23.404095 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38532552-35c9-4e91-a557-9a176b2d7f22-workload-socket\") pod \"e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx\" (UID: \"38532552-35c9-4e91-a557-9a176b2d7f22\") " pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" Jul 28 17:47:23.404421 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:23.404401 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38532552-35c9-4e91-a557-9a176b2d7f22-istio-data\") pod \"e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx\" (UID: \"38532552-35c9-4e91-a557-9a176b2d7f22\") " pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" Jul 28 17:47:23.404588 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:23.404433 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38532552-35c9-4e91-a557-9a176b2d7f22-workload-certs\") pod \"e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx\" (UID: \"38532552-35c9-4e91-a557-9a176b2d7f22\") " pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" Jul 28 17:47:23.404588 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:23.404518 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38532552-35c9-4e91-a557-9a176b2d7f22-credential-socket\") pod \"e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx\" (UID: \"38532552-35c9-4e91-a557-9a176b2d7f22\") " pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" Jul 28 17:47:23.404588 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:23.404559 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38532552-35c9-4e91-a557-9a176b2d7f22-workload-socket\") pod \"e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx\" (UID: \"38532552-35c9-4e91-a557-9a176b2d7f22\") " pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" Jul 28 17:47:23.405095 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:23.405067 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38532552-35c9-4e91-a557-9a176b2d7f22-istiod-ca-cert\") pod \"e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx\" (UID: \"38532552-35c9-4e91-a557-9a176b2d7f22\") " pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" Jul 28 17:47:23.406632 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:23.406580 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38532552-35c9-4e91-a557-9a176b2d7f22-istio-envoy\") pod \"e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx\" (UID: \"38532552-35c9-4e91-a557-9a176b2d7f22\") " pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" Jul 28 17:47:23.406718 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:23.406700 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38532552-35c9-4e91-a557-9a176b2d7f22-istio-podinfo\") pod \"e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx\" (UID: \"38532552-35c9-4e91-a557-9a176b2d7f22\") " pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" Jul 28 17:47:23.414166 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:23.414146 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38532552-35c9-4e91-a557-9a176b2d7f22-istio-token\") pod \"e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx\" (UID: \"38532552-35c9-4e91-a557-9a176b2d7f22\") " pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" Jul 28 17:47:23.414389 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:23.414372 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mn5hw\" (UniqueName: \"kubernetes.io/projected/38532552-35c9-4e91-a557-9a176b2d7f22-kube-api-access-mn5hw\") pod \"e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx\" (UID: \"38532552-35c9-4e91-a557-9a176b2d7f22\") " pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" Jul 28 17:47:23.471003 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:23.470969 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" Jul 28 17:47:23.631038 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:23.631006 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx"] Jul 28 17:47:23.633025 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:47:23.632153 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38532552_35c9_4e91_a557_9a176b2d7f22.slice/crio-73222d32c66e5100ec0ae20874ac79bb4e087b7d9360b81370d2b29db5c4e6e0 WatchSource:0}: Error finding container 73222d32c66e5100ec0ae20874ac79bb4e087b7d9360b81370d2b29db5c4e6e0: Status 404 returned error can't find the container with id 73222d32c66e5100ec0ae20874ac79bb4e087b7d9360b81370d2b29db5c4e6e0 Jul 28 17:47:23.635214 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:23.635176 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 17:47:23.635296 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:23.635258 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 17:47:23.635337 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:23.635297 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 17:47:24.460308 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:24.460275 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" event={"ID":"38532552-35c9-4e91-a557-9a176b2d7f22","Type":"ContainerStarted","Data":"83810d570b2f54f04322e5ed1ec155a54a4563c6ebb44310ae2f137dbcc74e39"} Jul 28 17:47:24.460720 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:24.460327 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" event={"ID":"38532552-35c9-4e91-a557-9a176b2d7f22","Type":"ContainerStarted","Data":"73222d32c66e5100ec0ae20874ac79bb4e087b7d9360b81370d2b29db5c4e6e0"} Jul 28 17:47:24.471952 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:24.471921 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" Jul 28 17:47:24.476711 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:24.476690 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" Jul 28 17:47:24.490392 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:24.490353 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" podStartSLOduration=1.490339237 podStartE2EDuration="1.490339237s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:47:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:47:24.489346498 +0000 UTC m=+2343.098277869" watchObservedRunningTime="2026-07-28 17:47:24.490339237 +0000 UTC m=+2343.099270606" Jul 28 17:47:25.465323 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:25.465289 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" Jul 28 17:47:25.466403 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:25.466383 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" Jul 28 17:47:40.466914 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:40.466880 2570 patch_prober.go:28] interesting pod/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Jul 28 17:47:40.467351 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:40.466936 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" podUID="38532552-35c9-4e91-a557-9a176b2d7f22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Jul 28 17:47:40.555762 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:40.555726 2570 generic.go:356] "Generic (PLEG): container finished" podID="38532552-35c9-4e91-a557-9a176b2d7f22" containerID="83810d570b2f54f04322e5ed1ec155a54a4563c6ebb44310ae2f137dbcc74e39" exitCode=0 Jul 28 17:47:40.555924 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:40.555775 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" event={"ID":"38532552-35c9-4e91-a557-9a176b2d7f22","Type":"ContainerDied","Data":"83810d570b2f54f04322e5ed1ec155a54a4563c6ebb44310ae2f137dbcc74e39"} Jul 28 17:47:40.556312 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:40.556294 2570 scope.go:117] "RemoveContainer" containerID="83810d570b2f54f04322e5ed1ec155a54a4563c6ebb44310ae2f137dbcc74e39" Jul 28 17:47:40.557442 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:40.557415 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 17:47:40.557508 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:40.557481 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 17:47:40.557546 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:40.557529 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 17:47:41.566135 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:41.566085 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" event={"ID":"38532552-35c9-4e91-a557-9a176b2d7f22","Type":"ContainerStarted","Data":"f30b06a787b989f7882ddff697dbbf3dd4970e9354bfc32e00460c01563452a4"} Jul 28 17:47:42.472166 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:42.472105 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" Jul 28 17:47:42.472518 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:42.472493 2570 patch_prober.go:28] interesting pod/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Jul 28 17:47:42.472581 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:42.472559 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" podUID="38532552-35c9-4e91-a557-9a176b2d7f22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Jul 28 17:47:43.472451 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:43.472412 2570 patch_prober.go:28] interesting pod/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Jul 28 17:47:43.472863 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:43.472493 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" podUID="38532552-35c9-4e91-a557-9a176b2d7f22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Jul 28 17:47:44.471456 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:44.471422 2570 patch_prober.go:28] interesting pod/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Jul 28 17:47:44.471626 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:44.471479 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" podUID="38532552-35c9-4e91-a557-9a176b2d7f22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Jul 28 17:47:45.494580 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:45.494500 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" Jul 28 17:47:45.494987 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:45.494789 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" Jul 28 17:47:45.495777 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:47:45.495755 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" Jul 28 17:48:22.051093 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:48:22.051061 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-slfqf_ecb82800-8cf0-4ef9-926c-155172fad7ff/console-operator/1.log" Jul 28 17:48:22.055902 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:48:22.055879 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-slfqf_ecb82800-8cf0-4ef9-926c-155172fad7ff/console-operator/1.log" Jul 28 17:51:22.433396 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:51:22.433363 2570 scope.go:117] "RemoveContainer" containerID="7a4ba09b9f5a89f60fd7d13541d59d3fdea7820e49c97bb65efcb1cb8f237033" Jul 28 17:51:22.443177 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:51:22.443152 2570 scope.go:117] "RemoveContainer" containerID="8a906c2f0eb18b7dd9a74dae5e27d396f70037702b44f543682c631f374ef7b9" Jul 28 17:51:22.450883 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:51:22.450864 2570 scope.go:117] "RemoveContainer" containerID="4273b3c3c450dfd35d6fbf191d9eb55f0311792c7ac92adfffaba5e30be94e88" Jul 28 17:52:22.476832 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:52:22.476796 2570 scope.go:117] "RemoveContainer" containerID="1a4df7b12672f7369e60ab3910a09d17a8519691f47db4ea09a79578856f113a" Jul 28 17:53:22.082938 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:53:22.082887 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-slfqf_ecb82800-8cf0-4ef9-926c-155172fad7ff/console-operator/1.log" Jul 28 17:53:22.089437 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:53:22.089413 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-slfqf_ecb82800-8cf0-4ef9-926c-155172fad7ff/console-operator/1.log" Jul 28 17:53:22.500493 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:53:22.500458 2570 scope.go:117] "RemoveContainer" containerID="daa3f53e3308fd2d193a4e61b82dcf95a1f473cf0b0b10ac0902826d1bb8301e" Jul 28 17:56:41.375419 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:41.375385 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7428-139971483966768"] Jul 28 17:56:41.379034 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:41.379016 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-139971483966768" Jul 28 17:56:41.381997 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:41.381979 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-cf5rx\"" Jul 28 17:56:41.388505 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:41.388478 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7428-139971483966768"] Jul 28 17:56:41.451487 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:41.451449 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbvmq\" (UniqueName: \"kubernetes.io/projected/44cbdcd7-751b-472f-ae94-8ef58bbdce84-kube-api-access-xbvmq\") pod \"test-curl-7428-139971483966768\" (UID: \"44cbdcd7-751b-472f-ae94-8ef58bbdce84\") " pod="opendatahub/test-curl-7428-139971483966768" Jul 28 17:56:41.552265 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:41.552233 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbvmq\" (UniqueName: \"kubernetes.io/projected/44cbdcd7-751b-472f-ae94-8ef58bbdce84-kube-api-access-xbvmq\") pod \"test-curl-7428-139971483966768\" (UID: \"44cbdcd7-751b-472f-ae94-8ef58bbdce84\") " pod="opendatahub/test-curl-7428-139971483966768" Jul 28 17:56:41.562427 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:41.562394 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbvmq\" (UniqueName: \"kubernetes.io/projected/44cbdcd7-751b-472f-ae94-8ef58bbdce84-kube-api-access-xbvmq\") pod \"test-curl-7428-139971483966768\" (UID: \"44cbdcd7-751b-472f-ae94-8ef58bbdce84\") " pod="opendatahub/test-curl-7428-139971483966768" Jul 28 17:56:41.689646 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:41.689556 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-139971483966768" Jul 28 17:56:41.837468 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:41.837443 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7428-139971483966768"] Jul 28 17:56:41.839872 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:56:41.839846 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod44cbdcd7_751b_472f_ae94_8ef58bbdce84.slice/crio-dda02b9e859cfcc70ae320b5cb798ad16fad9e0fb9827a5119dc2a41d6274dc5 WatchSource:0}: Error finding container dda02b9e859cfcc70ae320b5cb798ad16fad9e0fb9827a5119dc2a41d6274dc5: Status 404 returned error can't find the container with id dda02b9e859cfcc70ae320b5cb798ad16fad9e0fb9827a5119dc2a41d6274dc5 Jul 28 17:56:41.841612 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:41.841594 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 17:56:42.536682 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:42.536649 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7428-139971483966768" event={"ID":"44cbdcd7-751b-472f-ae94-8ef58bbdce84","Type":"ContainerStarted","Data":"dda02b9e859cfcc70ae320b5cb798ad16fad9e0fb9827a5119dc2a41d6274dc5"} Jul 28 17:56:43.544176 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:43.544103 2570 generic.go:356] "Generic (PLEG): container finished" podID="44cbdcd7-751b-472f-ae94-8ef58bbdce84" containerID="5e4bac59f2acd353fb49b8c57befd8d67e8b06d2c413a3a1acb80a9ad7fcdf60" exitCode=0 Jul 28 17:56:43.544566 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:43.544187 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7428-139971483966768" event={"ID":"44cbdcd7-751b-472f-ae94-8ef58bbdce84","Type":"ContainerDied","Data":"5e4bac59f2acd353fb49b8c57befd8d67e8b06d2c413a3a1acb80a9ad7fcdf60"} Jul 28 17:56:44.687784 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:44.687762 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-139971483966768" Jul 28 17:56:44.727104 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:44.727078 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7428-139971483966768_44cbdcd7-751b-472f-ae94-8ef58bbdce84/test-curl-7428-139971483966768/0.log" Jul 28 17:56:44.756248 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:44.756213 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7428-139971483966768"] Jul 28 17:56:44.758201 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:44.758178 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7428-139971483966768"] Jul 28 17:56:44.784649 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:44.784619 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xbvmq\" (UniqueName: \"kubernetes.io/projected/44cbdcd7-751b-472f-ae94-8ef58bbdce84-kube-api-access-xbvmq\") pod \"44cbdcd7-751b-472f-ae94-8ef58bbdce84\" (UID: \"44cbdcd7-751b-472f-ae94-8ef58bbdce84\") " Jul 28 17:56:44.786636 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:44.786607 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44cbdcd7-751b-472f-ae94-8ef58bbdce84-kube-api-access-xbvmq" (OuterVolumeSpecName: "kube-api-access-xbvmq") pod "44cbdcd7-751b-472f-ae94-8ef58bbdce84" (UID: "44cbdcd7-751b-472f-ae94-8ef58bbdce84"). InnerVolumeSpecName "kube-api-access-xbvmq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:56:44.875833 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:44.875801 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7428-139971483856944"] Jul 28 17:56:44.876240 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:44.876225 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44cbdcd7-751b-472f-ae94-8ef58bbdce84" containerName="test-curl-7428-139971483966768" Jul 28 17:56:44.876299 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:44.876241 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="44cbdcd7-751b-472f-ae94-8ef58bbdce84" containerName="test-curl-7428-139971483966768" Jul 28 17:56:44.876342 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:44.876325 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="44cbdcd7-751b-472f-ae94-8ef58bbdce84" containerName="test-curl-7428-139971483966768" Jul 28 17:56:44.879339 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:44.879321 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-139971483856944" Jul 28 17:56:44.886388 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:44.886362 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xbvmq\" (UniqueName: \"kubernetes.io/projected/44cbdcd7-751b-472f-ae94-8ef58bbdce84-kube-api-access-xbvmq\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:56:44.886927 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:44.886903 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7428-139971483856944"] Jul 28 17:56:44.987520 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:44.987487 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lblqr\" (UniqueName: \"kubernetes.io/projected/748f9e72-fece-4138-9b57-3691aa26b7fa-kube-api-access-lblqr\") pod \"test-curl-7428-139971483856944\" (UID: \"748f9e72-fece-4138-9b57-3691aa26b7fa\") " pod="opendatahub/test-curl-7428-139971483856944" Jul 28 17:56:45.088320 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:45.088281 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lblqr\" (UniqueName: \"kubernetes.io/projected/748f9e72-fece-4138-9b57-3691aa26b7fa-kube-api-access-lblqr\") pod \"test-curl-7428-139971483856944\" (UID: \"748f9e72-fece-4138-9b57-3691aa26b7fa\") " pod="opendatahub/test-curl-7428-139971483856944" Jul 28 17:56:45.098025 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:45.097986 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lblqr\" (UniqueName: \"kubernetes.io/projected/748f9e72-fece-4138-9b57-3691aa26b7fa-kube-api-access-lblqr\") pod \"test-curl-7428-139971483856944\" (UID: \"748f9e72-fece-4138-9b57-3691aa26b7fa\") " pod="opendatahub/test-curl-7428-139971483856944" Jul 28 17:56:45.189656 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:45.189554 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-139971483856944" Jul 28 17:56:45.326924 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:45.326895 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7428-139971483856944"] Jul 28 17:56:45.328446 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:56:45.328422 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod748f9e72_fece_4138_9b57_3691aa26b7fa.slice/crio-5a53e2a072dd333edc39834e1d19d0329dd3c293a5152328d832578025ffb1fe WatchSource:0}: Error finding container 5a53e2a072dd333edc39834e1d19d0329dd3c293a5152328d832578025ffb1fe: Status 404 returned error can't find the container with id 5a53e2a072dd333edc39834e1d19d0329dd3c293a5152328d832578025ffb1fe Jul 28 17:56:45.556662 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:45.556589 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-139971483966768" Jul 28 17:56:45.556662 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:45.556626 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dda02b9e859cfcc70ae320b5cb798ad16fad9e0fb9827a5119dc2a41d6274dc5" Jul 28 17:56:45.557894 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:45.557866 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7428-139971483856944" event={"ID":"748f9e72-fece-4138-9b57-3691aa26b7fa","Type":"ContainerStarted","Data":"5a53e2a072dd333edc39834e1d19d0329dd3c293a5152328d832578025ffb1fe"} Jul 28 17:56:45.569158 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:56:45.569102 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7428-139971483966768\" is forbidden: User \"system:node:ip-10-0-132-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-11.ec2.internal' and this object" podUID="44cbdcd7-751b-472f-ae94-8ef58bbdce84" pod="opendatahub/test-curl-7428-139971483966768" Jul 28 17:56:46.120514 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:46.120480 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44cbdcd7-751b-472f-ae94-8ef58bbdce84" path="/var/lib/kubelet/pods/44cbdcd7-751b-472f-ae94-8ef58bbdce84/volumes" Jul 28 17:56:46.565096 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:46.565016 2570 generic.go:356] "Generic (PLEG): container finished" podID="748f9e72-fece-4138-9b57-3691aa26b7fa" containerID="7e8c351e4af703ddefb49b0bd2c0d4a11ea4716089df474dc244cecb224a26c7" exitCode=0 Jul 28 17:56:46.565096 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:46.565057 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7428-139971483856944" event={"ID":"748f9e72-fece-4138-9b57-3691aa26b7fa","Type":"ContainerDied","Data":"7e8c351e4af703ddefb49b0bd2c0d4a11ea4716089df474dc244cecb224a26c7"} Jul 28 17:56:47.707976 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:47.707955 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-139971483856944" Jul 28 17:56:47.732430 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:47.732395 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7428-139971483856944_748f9e72-fece-4138-9b57-3691aa26b7fa/test-curl-7428-139971483856944/0.log" Jul 28 17:56:47.763391 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:47.763351 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7428-139971483856944"] Jul 28 17:56:47.768409 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:47.768383 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7428-139971483856944"] Jul 28 17:56:47.817650 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:47.817621 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lblqr\" (UniqueName: \"kubernetes.io/projected/748f9e72-fece-4138-9b57-3691aa26b7fa-kube-api-access-lblqr\") pod \"748f9e72-fece-4138-9b57-3691aa26b7fa\" (UID: \"748f9e72-fece-4138-9b57-3691aa26b7fa\") " Jul 28 17:56:47.819640 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:47.819607 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/748f9e72-fece-4138-9b57-3691aa26b7fa-kube-api-access-lblqr" (OuterVolumeSpecName: "kube-api-access-lblqr") pod "748f9e72-fece-4138-9b57-3691aa26b7fa" (UID: "748f9e72-fece-4138-9b57-3691aa26b7fa"). InnerVolumeSpecName "kube-api-access-lblqr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:56:47.919056 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:47.919024 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lblqr\" (UniqueName: \"kubernetes.io/projected/748f9e72-fece-4138-9b57-3691aa26b7fa-kube-api-access-lblqr\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:56:47.925914 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:47.925884 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7428-139971483940016"] Jul 28 17:56:47.926368 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:47.926351 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="748f9e72-fece-4138-9b57-3691aa26b7fa" containerName="test-curl-7428-139971483856944" Jul 28 17:56:47.926461 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:47.926370 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="748f9e72-fece-4138-9b57-3691aa26b7fa" containerName="test-curl-7428-139971483856944" Jul 28 17:56:47.926518 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:47.926487 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="748f9e72-fece-4138-9b57-3691aa26b7fa" containerName="test-curl-7428-139971483856944" Jul 28 17:56:47.929516 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:47.929498 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-139971483940016" Jul 28 17:56:47.940063 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:47.940035 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7428-139971483940016"] Jul 28 17:56:48.019610 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:48.019573 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6nsd\" (UniqueName: \"kubernetes.io/projected/a5c53ad2-282e-4739-8c0a-99237ba490fe-kube-api-access-b6nsd\") pod \"test-curl-7428-139971483940016\" (UID: \"a5c53ad2-282e-4739-8c0a-99237ba490fe\") " pod="opendatahub/test-curl-7428-139971483940016" Jul 28 17:56:48.119904 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:48.119865 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="748f9e72-fece-4138-9b57-3691aa26b7fa" path="/var/lib/kubelet/pods/748f9e72-fece-4138-9b57-3691aa26b7fa/volumes" Jul 28 17:56:48.120202 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:48.120156 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6nsd\" (UniqueName: \"kubernetes.io/projected/a5c53ad2-282e-4739-8c0a-99237ba490fe-kube-api-access-b6nsd\") pod \"test-curl-7428-139971483940016\" (UID: \"a5c53ad2-282e-4739-8c0a-99237ba490fe\") " pod="opendatahub/test-curl-7428-139971483940016" Jul 28 17:56:48.130369 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:48.130346 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6nsd\" (UniqueName: \"kubernetes.io/projected/a5c53ad2-282e-4739-8c0a-99237ba490fe-kube-api-access-b6nsd\") pod \"test-curl-7428-139971483940016\" (UID: \"a5c53ad2-282e-4739-8c0a-99237ba490fe\") " pod="opendatahub/test-curl-7428-139971483940016" Jul 28 17:56:48.240727 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:48.240687 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-139971483940016" Jul 28 17:56:48.378277 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:48.378253 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7428-139971483940016"] Jul 28 17:56:48.379815 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:56:48.379786 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda5c53ad2_282e_4739_8c0a_99237ba490fe.slice/crio-679f8c310f068a289a5046c704c59eff967796192e706aa32405c4acf853d586 WatchSource:0}: Error finding container 679f8c310f068a289a5046c704c59eff967796192e706aa32405c4acf853d586: Status 404 returned error can't find the container with id 679f8c310f068a289a5046c704c59eff967796192e706aa32405c4acf853d586 Jul 28 17:56:48.576275 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:48.576237 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7428-139971483940016" event={"ID":"a5c53ad2-282e-4739-8c0a-99237ba490fe","Type":"ContainerStarted","Data":"679f8c310f068a289a5046c704c59eff967796192e706aa32405c4acf853d586"} Jul 28 17:56:48.577532 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:48.577507 2570 scope.go:117] "RemoveContainer" containerID="7e8c351e4af703ddefb49b0bd2c0d4a11ea4716089df474dc244cecb224a26c7" Jul 28 17:56:48.577677 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:48.577513 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-139971483856944" Jul 28 17:56:49.585661 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:49.585623 2570 generic.go:356] "Generic (PLEG): container finished" podID="a5c53ad2-282e-4739-8c0a-99237ba490fe" containerID="bd676dc9d612e32e6b0eaabfaa44ff55a74898f05d6bf33ebf801f84d62aa2e8" exitCode=0 Jul 28 17:56:49.586019 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:49.585707 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7428-139971483940016" event={"ID":"a5c53ad2-282e-4739-8c0a-99237ba490fe","Type":"ContainerDied","Data":"bd676dc9d612e32e6b0eaabfaa44ff55a74898f05d6bf33ebf801f84d62aa2e8"} Jul 28 17:56:50.738124 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:50.738083 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-139971483940016" Jul 28 17:56:50.777211 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:50.777184 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7428-139971483940016_a5c53ad2-282e-4739-8c0a-99237ba490fe/test-curl-7428-139971483940016/0.log" Jul 28 17:56:50.807825 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:50.807793 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7428-139971483940016"] Jul 28 17:56:50.815352 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:50.815325 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7428-139971483940016"] Jul 28 17:56:50.845687 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:50.845601 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b6nsd\" (UniqueName: \"kubernetes.io/projected/a5c53ad2-282e-4739-8c0a-99237ba490fe-kube-api-access-b6nsd\") pod \"a5c53ad2-282e-4739-8c0a-99237ba490fe\" (UID: \"a5c53ad2-282e-4739-8c0a-99237ba490fe\") " Jul 28 17:56:50.847633 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:50.847604 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5c53ad2-282e-4739-8c0a-99237ba490fe-kube-api-access-b6nsd" (OuterVolumeSpecName: "kube-api-access-b6nsd") pod "a5c53ad2-282e-4739-8c0a-99237ba490fe" (UID: "a5c53ad2-282e-4739-8c0a-99237ba490fe"). InnerVolumeSpecName "kube-api-access-b6nsd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:56:50.943192 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:50.943154 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7428-139971483940016"] Jul 28 17:56:50.943826 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:50.943808 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5c53ad2-282e-4739-8c0a-99237ba490fe" containerName="test-curl-7428-139971483940016" Jul 28 17:56:50.943865 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:50.943830 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5c53ad2-282e-4739-8c0a-99237ba490fe" containerName="test-curl-7428-139971483940016" Jul 28 17:56:50.943937 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:50.943926 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5c53ad2-282e-4739-8c0a-99237ba490fe" containerName="test-curl-7428-139971483940016" Jul 28 17:56:50.946995 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:50.946969 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6nsd\" (UniqueName: \"kubernetes.io/projected/a5c53ad2-282e-4739-8c0a-99237ba490fe-kube-api-access-b6nsd\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:56:50.946995 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:50.946978 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-139971483940016" Jul 28 17:56:50.952959 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:50.952931 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7428-139971483940016"] Jul 28 17:56:51.048182 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:51.048142 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qv4rb\" (UniqueName: \"kubernetes.io/projected/e9421b87-3907-420f-9362-e79d96c9a1b0-kube-api-access-qv4rb\") pod \"test-curl-7428-139971483940016\" (UID: \"e9421b87-3907-420f-9362-e79d96c9a1b0\") " pod="opendatahub/test-curl-7428-139971483940016" Jul 28 17:56:51.149088 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:51.149054 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv4rb\" (UniqueName: \"kubernetes.io/projected/e9421b87-3907-420f-9362-e79d96c9a1b0-kube-api-access-qv4rb\") pod \"test-curl-7428-139971483940016\" (UID: \"e9421b87-3907-420f-9362-e79d96c9a1b0\") " pod="opendatahub/test-curl-7428-139971483940016" Jul 28 17:56:51.159043 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:51.159011 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qv4rb\" (UniqueName: \"kubernetes.io/projected/e9421b87-3907-420f-9362-e79d96c9a1b0-kube-api-access-qv4rb\") pod \"test-curl-7428-139971483940016\" (UID: \"e9421b87-3907-420f-9362-e79d96c9a1b0\") " pod="opendatahub/test-curl-7428-139971483940016" Jul 28 17:56:51.257941 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:51.257912 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-139971483940016" Jul 28 17:56:51.393510 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:51.393480 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7428-139971483940016"] Jul 28 17:56:51.394737 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:56:51.394714 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode9421b87_3907_420f_9362_e79d96c9a1b0.slice/crio-6a9fa16052ecfad5ade137d2ed037b4bc8201271737dae36c2abae33fd453ed7 WatchSource:0}: Error finding container 6a9fa16052ecfad5ade137d2ed037b4bc8201271737dae36c2abae33fd453ed7: Status 404 returned error can't find the container with id 6a9fa16052ecfad5ade137d2ed037b4bc8201271737dae36c2abae33fd453ed7 Jul 28 17:56:51.597999 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:51.597957 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7428-139971483940016" event={"ID":"e9421b87-3907-420f-9362-e79d96c9a1b0","Type":"ContainerStarted","Data":"6a9fa16052ecfad5ade137d2ed037b4bc8201271737dae36c2abae33fd453ed7"} Jul 28 17:56:51.599212 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:51.599190 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="679f8c310f068a289a5046c704c59eff967796192e706aa32405c4acf853d586" Jul 28 17:56:51.599316 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:51.599212 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-139971483940016" Jul 28 17:56:51.612075 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:51.612045 2570 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7428-139971483940016" oldPodUID="a5c53ad2-282e-4739-8c0a-99237ba490fe" podUID="e9421b87-3907-420f-9362-e79d96c9a1b0" Jul 28 17:56:52.121541 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:52.121512 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5c53ad2-282e-4739-8c0a-99237ba490fe" path="/var/lib/kubelet/pods/a5c53ad2-282e-4739-8c0a-99237ba490fe/volumes" Jul 28 17:56:52.607166 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:52.607070 2570 generic.go:356] "Generic (PLEG): container finished" podID="e9421b87-3907-420f-9362-e79d96c9a1b0" containerID="97b646718dbfa74e4d2b1f57e788bb28d47069dfdbef958f9972b3eb959e143c" exitCode=0 Jul 28 17:56:52.607302 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:52.607161 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7428-139971483940016" event={"ID":"e9421b87-3907-420f-9362-e79d96c9a1b0","Type":"ContainerDied","Data":"97b646718dbfa74e4d2b1f57e788bb28d47069dfdbef958f9972b3eb959e143c"} Jul 28 17:56:53.765778 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:53.765753 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-139971483940016" Jul 28 17:56:53.788693 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:53.788655 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7428-139971483940016_e9421b87-3907-420f-9362-e79d96c9a1b0/test-curl-7428-139971483940016/0.log" Jul 28 17:56:53.815943 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:53.815912 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7428-139971483940016"] Jul 28 17:56:53.820254 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:53.820224 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7428-139971483940016"] Jul 28 17:56:53.876425 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:53.876394 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qv4rb\" (UniqueName: \"kubernetes.io/projected/e9421b87-3907-420f-9362-e79d96c9a1b0-kube-api-access-qv4rb\") pod \"e9421b87-3907-420f-9362-e79d96c9a1b0\" (UID: \"e9421b87-3907-420f-9362-e79d96c9a1b0\") " Jul 28 17:56:53.878495 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:53.878470 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9421b87-3907-420f-9362-e79d96c9a1b0-kube-api-access-qv4rb" (OuterVolumeSpecName: "kube-api-access-qv4rb") pod "e9421b87-3907-420f-9362-e79d96c9a1b0" (UID: "e9421b87-3907-420f-9362-e79d96c9a1b0"). InnerVolumeSpecName "kube-api-access-qv4rb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:56:53.947787 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:53.947753 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7428-139971483941024"] Jul 28 17:56:53.948207 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:53.948191 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9421b87-3907-420f-9362-e79d96c9a1b0" containerName="test-curl-7428-139971483940016" Jul 28 17:56:53.948207 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:53.948207 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9421b87-3907-420f-9362-e79d96c9a1b0" containerName="test-curl-7428-139971483940016" Jul 28 17:56:53.948290 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:53.948283 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9421b87-3907-420f-9362-e79d96c9a1b0" containerName="test-curl-7428-139971483940016" Jul 28 17:56:53.951329 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:53.951308 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-139971483941024" Jul 28 17:56:53.959448 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:53.959421 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7428-139971483941024"] Jul 28 17:56:53.977611 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:53.977573 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qv4rb\" (UniqueName: \"kubernetes.io/projected/e9421b87-3907-420f-9362-e79d96c9a1b0-kube-api-access-qv4rb\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:56:54.078787 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:54.078748 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86kbt\" (UniqueName: \"kubernetes.io/projected/3af9c80d-e768-4077-8bda-18d54b86f5fa-kube-api-access-86kbt\") pod \"test-curl-7428-139971483941024\" (UID: \"3af9c80d-e768-4077-8bda-18d54b86f5fa\") " pod="opendatahub/test-curl-7428-139971483941024" Jul 28 17:56:54.120627 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:54.120597 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9421b87-3907-420f-9362-e79d96c9a1b0" path="/var/lib/kubelet/pods/e9421b87-3907-420f-9362-e79d96c9a1b0/volumes" Jul 28 17:56:54.179391 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:54.179353 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86kbt\" (UniqueName: \"kubernetes.io/projected/3af9c80d-e768-4077-8bda-18d54b86f5fa-kube-api-access-86kbt\") pod \"test-curl-7428-139971483941024\" (UID: \"3af9c80d-e768-4077-8bda-18d54b86f5fa\") " pod="opendatahub/test-curl-7428-139971483941024" Jul 28 17:56:54.188847 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:54.188819 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86kbt\" (UniqueName: \"kubernetes.io/projected/3af9c80d-e768-4077-8bda-18d54b86f5fa-kube-api-access-86kbt\") pod \"test-curl-7428-139971483941024\" (UID: \"3af9c80d-e768-4077-8bda-18d54b86f5fa\") " pod="opendatahub/test-curl-7428-139971483941024" Jul 28 17:56:54.263247 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:54.263209 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-139971483941024" Jul 28 17:56:54.404517 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:54.404491 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7428-139971483941024"] Jul 28 17:56:54.406567 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:56:54.406539 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3af9c80d_e768_4077_8bda_18d54b86f5fa.slice/crio-6bc4eadedc9be15d564b350476be8931c13d2e873d1d9e8de62536c6a8aca795 WatchSource:0}: Error finding container 6bc4eadedc9be15d564b350476be8931c13d2e873d1d9e8de62536c6a8aca795: Status 404 returned error can't find the container with id 6bc4eadedc9be15d564b350476be8931c13d2e873d1d9e8de62536c6a8aca795 Jul 28 17:56:54.622809 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:54.622785 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-139971483940016" Jul 28 17:56:54.623005 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:54.622830 2570 scope.go:117] "RemoveContainer" containerID="97b646718dbfa74e4d2b1f57e788bb28d47069dfdbef958f9972b3eb959e143c" Jul 28 17:56:54.624155 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:54.624106 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7428-139971483941024" event={"ID":"3af9c80d-e768-4077-8bda-18d54b86f5fa","Type":"ContainerStarted","Data":"6bc4eadedc9be15d564b350476be8931c13d2e873d1d9e8de62536c6a8aca795"} Jul 28 17:56:55.630732 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:55.630691 2570 generic.go:356] "Generic (PLEG): container finished" podID="3af9c80d-e768-4077-8bda-18d54b86f5fa" containerID="f81be337e19ac20cde0f43563e4b3365b1e336810124e4d4ad5a32e2e0298c4c" exitCode=0 Jul 28 17:56:55.631174 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:55.630788 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7428-139971483941024" event={"ID":"3af9c80d-e768-4077-8bda-18d54b86f5fa","Type":"ContainerDied","Data":"f81be337e19ac20cde0f43563e4b3365b1e336810124e4d4ad5a32e2e0298c4c"} Jul 28 17:56:56.778522 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:56.778493 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-139971483941024" Jul 28 17:56:56.804347 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:56.804317 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7428-139971483941024_3af9c80d-e768-4077-8bda-18d54b86f5fa/test-curl-7428-139971483941024/0.log" Jul 28 17:56:56.830926 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:56.830891 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7428-139971483941024"] Jul 28 17:56:56.833860 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:56.833834 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7428-139971483941024"] Jul 28 17:56:56.906553 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:56.906465 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-86kbt\" (UniqueName: \"kubernetes.io/projected/3af9c80d-e768-4077-8bda-18d54b86f5fa-kube-api-access-86kbt\") pod \"3af9c80d-e768-4077-8bda-18d54b86f5fa\" (UID: \"3af9c80d-e768-4077-8bda-18d54b86f5fa\") " Jul 28 17:56:56.908545 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:56.908513 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3af9c80d-e768-4077-8bda-18d54b86f5fa-kube-api-access-86kbt" (OuterVolumeSpecName: "kube-api-access-86kbt") pod "3af9c80d-e768-4077-8bda-18d54b86f5fa" (UID: "3af9c80d-e768-4077-8bda-18d54b86f5fa"). InnerVolumeSpecName "kube-api-access-86kbt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:56:57.007358 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:57.007325 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86kbt\" (UniqueName: \"kubernetes.io/projected/3af9c80d-e768-4077-8bda-18d54b86f5fa-kube-api-access-86kbt\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:56:57.642765 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:57.642733 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-139971483941024" Jul 28 17:56:57.642765 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:57.642767 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6bc4eadedc9be15d564b350476be8931c13d2e873d1d9e8de62536c6a8aca795" Jul 28 17:56:57.654254 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:56:57.654219 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7428-139971483941024\" is forbidden: User \"system:node:ip-10-0-132-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-11.ec2.internal' and this object" podUID="3af9c80d-e768-4077-8bda-18d54b86f5fa" pod="opendatahub/test-curl-7428-139971483941024" Jul 28 17:56:58.120028 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:56:58.120001 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3af9c80d-e768-4077-8bda-18d54b86f5fa" path="/var/lib/kubelet/pods/3af9c80d-e768-4077-8bda-18d54b86f5fa/volumes" Jul 28 17:57:19.561250 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:19.561213 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt"] Jul 28 17:57:19.561912 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:19.561681 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3af9c80d-e768-4077-8bda-18d54b86f5fa" containerName="test-curl-7428-139971483941024" Jul 28 17:57:19.561912 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:19.561695 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="3af9c80d-e768-4077-8bda-18d54b86f5fa" containerName="test-curl-7428-139971483941024" Jul 28 17:57:19.561912 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:19.561750 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="3af9c80d-e768-4077-8bda-18d54b86f5fa" containerName="test-curl-7428-139971483941024" Jul 28 17:57:19.566246 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:19.566222 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" Jul 28 17:57:19.569390 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:19.569365 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-857ec3-openshift-default-dockercfg-xwdpt\"" Jul 28 17:57:19.584657 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:19.584633 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt"] Jul 28 17:57:19.618784 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:19.618748 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-workload-certs\") pod \"e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt\" (UID: \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\") " pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" Jul 28 17:57:19.618908 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:19.618797 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-istio-envoy\") pod \"e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt\" (UID: \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\") " pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" Jul 28 17:57:19.618908 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:19.618856 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-workload-socket\") pod \"e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt\" (UID: \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\") " pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" Jul 28 17:57:19.618908 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:19.618883 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-istio-data\") pod \"e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt\" (UID: \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\") " pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" Jul 28 17:57:19.619013 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:19.618962 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-istiod-ca-cert\") pod \"e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt\" (UID: \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\") " pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" Jul 28 17:57:19.619052 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:19.619024 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-credential-socket\") pod \"e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt\" (UID: \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\") " pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" Jul 28 17:57:19.619087 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:19.619061 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-istio-token\") pod \"e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt\" (UID: \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\") " pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" Jul 28 17:57:19.619172 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:19.619159 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wf57x\" (UniqueName: \"kubernetes.io/projected/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-kube-api-access-wf57x\") pod \"e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt\" (UID: \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\") " pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" Jul 28 17:57:19.619218 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:19.619182 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-istio-podinfo\") pod \"e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt\" (UID: \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\") " pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" Jul 28 17:57:19.719738 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:19.719695 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-istiod-ca-cert\") pod \"e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt\" (UID: \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\") " pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" Jul 28 17:57:19.719932 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:19.719747 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-credential-socket\") pod \"e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt\" (UID: \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\") " pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" Jul 28 17:57:19.719932 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:19.719785 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-istio-token\") pod \"e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt\" (UID: \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\") " pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" Jul 28 17:57:19.719932 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:19.719838 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wf57x\" (UniqueName: \"kubernetes.io/projected/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-kube-api-access-wf57x\") pod \"e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt\" (UID: \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\") " pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" Jul 28 17:57:19.719932 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:19.719888 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-istio-podinfo\") pod \"e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt\" (UID: \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\") " pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" Jul 28 17:57:19.720208 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:19.719938 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-workload-certs\") pod \"e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt\" (UID: \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\") " pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" Jul 28 17:57:19.720208 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:19.719973 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-istio-envoy\") pod \"e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt\" (UID: \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\") " pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" Jul 28 17:57:19.720208 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:19.720023 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-workload-socket\") pod \"e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt\" (UID: \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\") " pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" Jul 28 17:57:19.720208 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:19.720066 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-istio-data\") pod \"e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt\" (UID: \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\") " pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" Jul 28 17:57:19.720431 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:19.720345 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-workload-certs\") pod \"e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt\" (UID: \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\") " pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" Jul 28 17:57:19.720499 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:19.720470 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-istio-data\") pod \"e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt\" (UID: \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\") " pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" Jul 28 17:57:19.720558 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:19.720508 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-istiod-ca-cert\") pod \"e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt\" (UID: \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\") " pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" Jul 28 17:57:19.720558 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:19.720527 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-credential-socket\") pod \"e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt\" (UID: \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\") " pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" Jul 28 17:57:19.720638 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:19.720529 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-workload-socket\") pod \"e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt\" (UID: \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\") " pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" Jul 28 17:57:19.722309 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:19.722285 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-istio-envoy\") pod \"e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt\" (UID: \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\") " pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" Jul 28 17:57:19.722492 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:19.722473 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-istio-podinfo\") pod \"e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt\" (UID: \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\") " pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" Jul 28 17:57:19.729626 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:19.729604 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wf57x\" (UniqueName: \"kubernetes.io/projected/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-kube-api-access-wf57x\") pod \"e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt\" (UID: \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\") " pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" Jul 28 17:57:19.731408 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:19.731384 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-istio-token\") pod \"e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt\" (UID: \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\") " pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" Jul 28 17:57:19.876639 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:19.876602 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" Jul 28 17:57:20.035372 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:20.035326 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt"] Jul 28 17:57:20.036432 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:57:20.036405 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a5389c9_9f5d_4cbf_a61a_26c1871bfbca.slice/crio-261ffb54cc683c382440870475555a5db20d7981425eb7ee10c697e3f7db83cb WatchSource:0}: Error finding container 261ffb54cc683c382440870475555a5db20d7981425eb7ee10c697e3f7db83cb: Status 404 returned error can't find the container with id 261ffb54cc683c382440870475555a5db20d7981425eb7ee10c697e3f7db83cb Jul 28 17:57:20.038709 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:20.038673 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 17:57:20.038810 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:20.038738 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 17:57:20.038810 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:20.038773 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 17:57:20.768828 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:20.768794 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" event={"ID":"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca","Type":"ContainerStarted","Data":"95b577752a53116f83271f060fce34748f3ba111d184d7a0239ce282606b0c5d"} Jul 28 17:57:20.769223 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:20.768845 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" event={"ID":"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca","Type":"ContainerStarted","Data":"261ffb54cc683c382440870475555a5db20d7981425eb7ee10c697e3f7db83cb"} Jul 28 17:57:20.833332 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:20.833280 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" podStartSLOduration=1.833263297 podStartE2EDuration="1.833263297s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:57:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:57:20.831594474 +0000 UTC m=+2939.440525846" watchObservedRunningTime="2026-07-28 17:57:20.833263297 +0000 UTC m=+2939.442194738" Jul 28 17:57:20.877666 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:20.877632 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" Jul 28 17:57:21.882266 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:21.882235 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" Jul 28 17:57:22.779146 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:22.779099 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" Jul 28 17:57:22.779978 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:22.779958 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" Jul 28 17:57:38.821266 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:38.821231 2570 patch_prober.go:28] interesting pod/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 28 17:57:38.821638 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:38.821301 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" podUID="5a5389c9-9f5d-4cbf-a61a-26c1871bfbca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 28 17:57:39.891052 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:39.891013 2570 generic.go:356] "Generic (PLEG): container finished" podID="5a5389c9-9f5d-4cbf-a61a-26c1871bfbca" containerID="95b577752a53116f83271f060fce34748f3ba111d184d7a0239ce282606b0c5d" exitCode=0 Jul 28 17:57:39.894476 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:39.891156 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" event={"ID":"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca","Type":"ContainerDied","Data":"95b577752a53116f83271f060fce34748f3ba111d184d7a0239ce282606b0c5d"} Jul 28 17:57:39.894476 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:39.892338 2570 scope.go:117] "RemoveContainer" containerID="95b577752a53116f83271f060fce34748f3ba111d184d7a0239ce282606b0c5d" Jul 28 17:57:39.895380 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:39.895348 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 17:57:39.895480 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:39.895422 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 17:57:39.895480 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:39.895455 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 17:57:40.899512 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:40.899476 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" event={"ID":"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca","Type":"ContainerStarted","Data":"5e3a7d56012b8846ec8ab531388afc0db150a2b240533e146fc993385f699540"} Jul 28 17:57:41.877484 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:41.877443 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" Jul 28 17:57:41.877702 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:41.877654 2570 patch_prober.go:28] interesting pod/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Jul 28 17:57:41.877784 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:41.877727 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" podUID="5a5389c9-9f5d-4cbf-a61a-26c1871bfbca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Jul 28 17:57:42.877390 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:42.877356 2570 patch_prober.go:28] interesting pod/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Jul 28 17:57:42.877782 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:42.877411 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" podUID="5a5389c9-9f5d-4cbf-a61a-26c1871bfbca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Jul 28 17:57:43.877292 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:43.877258 2570 patch_prober.go:28] interesting pod/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Jul 28 17:57:43.877468 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:43.877312 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" podUID="5a5389c9-9f5d-4cbf-a61a-26c1871bfbca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Jul 28 17:57:44.877265 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:44.877229 2570 patch_prober.go:28] interesting pod/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Jul 28 17:57:44.877438 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:44.877282 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" podUID="5a5389c9-9f5d-4cbf-a61a-26c1871bfbca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Jul 28 17:57:45.880360 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:45.880323 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" Jul 28 17:57:45.880825 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:45.880716 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" Jul 28 17:57:45.881307 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:57:45.881285 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" Jul 28 17:58:22.116417 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:58:22.116382 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-slfqf_ecb82800-8cf0-4ef9-926c-155172fad7ff/console-operator/1.log" Jul 28 17:58:22.126358 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:58:22.126330 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-slfqf_ecb82800-8cf0-4ef9-926c-155172fad7ff/console-operator/1.log" Jul 28 17:59:00.776123 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:00.776021 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-7fdc90-855bd8bcc4-jvpt7"] Jul 28 17:59:00.779969 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:00.779947 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7fdc90-855bd8bcc4-jvpt7" Jul 28 17:59:00.782623 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:00.782597 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-7fdc90\"" Jul 28 17:59:00.782728 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:00.782708 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-7fdc90-dockercfg-nflhq\"" Jul 28 17:59:00.791385 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:00.791364 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-7fdc90-855bd8bcc4-jvpt7"] Jul 28 17:59:00.937663 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:00.937629 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tjkz\" (UniqueName: \"kubernetes.io/projected/6882b4e8-cb9e-46ea-8b21-bb47cb2ffb8c-kube-api-access-8tjkz\") pod \"payload-processing-e2e-ipp-cleanup-7fdc90-855bd8bcc4-jvpt7\" (UID: \"6882b4e8-cb9e-46ea-8b21-bb47cb2ffb8c\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7fdc90-855bd8bcc4-jvpt7" Jul 28 17:59:00.937836 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:00.937802 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6882b4e8-cb9e-46ea-8b21-bb47cb2ffb8c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-7fdc90-855bd8bcc4-jvpt7\" (UID: \"6882b4e8-cb9e-46ea-8b21-bb47cb2ffb8c\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7fdc90-855bd8bcc4-jvpt7" Jul 28 17:59:01.038843 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:01.038751 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tjkz\" (UniqueName: \"kubernetes.io/projected/6882b4e8-cb9e-46ea-8b21-bb47cb2ffb8c-kube-api-access-8tjkz\") pod \"payload-processing-e2e-ipp-cleanup-7fdc90-855bd8bcc4-jvpt7\" (UID: \"6882b4e8-cb9e-46ea-8b21-bb47cb2ffb8c\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7fdc90-855bd8bcc4-jvpt7" Jul 28 17:59:01.039215 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:01.039180 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6882b4e8-cb9e-46ea-8b21-bb47cb2ffb8c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-7fdc90-855bd8bcc4-jvpt7\" (UID: \"6882b4e8-cb9e-46ea-8b21-bb47cb2ffb8c\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7fdc90-855bd8bcc4-jvpt7" Jul 28 17:59:01.039945 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:01.039923 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6882b4e8-cb9e-46ea-8b21-bb47cb2ffb8c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-7fdc90-855bd8bcc4-jvpt7\" (UID: \"6882b4e8-cb9e-46ea-8b21-bb47cb2ffb8c\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7fdc90-855bd8bcc4-jvpt7" Jul 28 17:59:01.048850 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:01.048821 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tjkz\" (UniqueName: \"kubernetes.io/projected/6882b4e8-cb9e-46ea-8b21-bb47cb2ffb8c-kube-api-access-8tjkz\") pod \"payload-processing-e2e-ipp-cleanup-7fdc90-855bd8bcc4-jvpt7\" (UID: \"6882b4e8-cb9e-46ea-8b21-bb47cb2ffb8c\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7fdc90-855bd8bcc4-jvpt7" Jul 28 17:59:01.091908 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:01.091876 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7fdc90-855bd8bcc4-jvpt7" Jul 28 17:59:01.243804 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:01.243778 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-7fdc90-855bd8bcc4-jvpt7"] Jul 28 17:59:01.245778 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:59:01.245752 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6882b4e8_cb9e_46ea_8b21_bb47cb2ffb8c.slice/crio-3c1e542c23c82130d6c11bde694aebd836aae932aa5f55e908424d58b6c5be3e WatchSource:0}: Error finding container 3c1e542c23c82130d6c11bde694aebd836aae932aa5f55e908424d58b6c5be3e: Status 404 returned error can't find the container with id 3c1e542c23c82130d6c11bde694aebd836aae932aa5f55e908424d58b6c5be3e Jul 28 17:59:01.342193 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:01.342153 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7fdc90-855bd8bcc4-jvpt7" event={"ID":"6882b4e8-cb9e-46ea-8b21-bb47cb2ffb8c","Type":"ContainerStarted","Data":"e859f148cff679694f2841d999c6af94aeea17787712660d4273bd00cec8921e"} Jul 28 17:59:01.342371 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:01.342351 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7fdc90-855bd8bcc4-jvpt7" Jul 28 17:59:01.342436 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:01.342389 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7fdc90-855bd8bcc4-jvpt7" event={"ID":"6882b4e8-cb9e-46ea-8b21-bb47cb2ffb8c","Type":"ContainerStarted","Data":"3c1e542c23c82130d6c11bde694aebd836aae932aa5f55e908424d58b6c5be3e"} Jul 28 17:59:01.366223 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:01.366163 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7fdc90-855bd8bcc4-jvpt7" podStartSLOduration=1.3661435069999999 podStartE2EDuration="1.366143507s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:59:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:59:01.364367679 +0000 UTC m=+3039.973299066" watchObservedRunningTime="2026-07-28 17:59:01.366143507 +0000 UTC m=+3039.975074870" Jul 28 17:59:12.353726 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:12.353694 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7fdc90-855bd8bcc4-jvpt7" Jul 28 17:59:31.170790 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:31.170751 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-7fdc90-855bd8bcc4-jvpt7"] Jul 28 17:59:31.171643 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:31.171588 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7fdc90-855bd8bcc4-jvpt7" podUID="6882b4e8-cb9e-46ea-8b21-bb47cb2ffb8c" containerName="payload-processing" containerID="cri-o://e859f148cff679694f2841d999c6af94aeea17787712660d4273bd00cec8921e" gracePeriod=30 Jul 28 17:59:31.434857 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:31.434781 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7fdc90-855bd8bcc4-jvpt7" Jul 28 17:59:31.523452 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:31.523418 2570 generic.go:356] "Generic (PLEG): container finished" podID="6882b4e8-cb9e-46ea-8b21-bb47cb2ffb8c" containerID="e859f148cff679694f2841d999c6af94aeea17787712660d4273bd00cec8921e" exitCode=0 Jul 28 17:59:31.523622 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:31.523462 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7fdc90-855bd8bcc4-jvpt7" Jul 28 17:59:31.523622 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:31.523486 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7fdc90-855bd8bcc4-jvpt7" event={"ID":"6882b4e8-cb9e-46ea-8b21-bb47cb2ffb8c","Type":"ContainerDied","Data":"e859f148cff679694f2841d999c6af94aeea17787712660d4273bd00cec8921e"} Jul 28 17:59:31.523622 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:31.523527 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7fdc90-855bd8bcc4-jvpt7" event={"ID":"6882b4e8-cb9e-46ea-8b21-bb47cb2ffb8c","Type":"ContainerDied","Data":"3c1e542c23c82130d6c11bde694aebd836aae932aa5f55e908424d58b6c5be3e"} Jul 28 17:59:31.523622 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:31.523548 2570 scope.go:117] "RemoveContainer" containerID="e859f148cff679694f2841d999c6af94aeea17787712660d4273bd00cec8921e" Jul 28 17:59:31.532589 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:31.532565 2570 scope.go:117] "RemoveContainer" containerID="e859f148cff679694f2841d999c6af94aeea17787712660d4273bd00cec8921e" Jul 28 17:59:31.532924 ip-10-0-132-11 kubenswrapper[2570]: E0728 17:59:31.532903 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e859f148cff679694f2841d999c6af94aeea17787712660d4273bd00cec8921e\": container with ID starting with e859f148cff679694f2841d999c6af94aeea17787712660d4273bd00cec8921e not found: ID does not exist" containerID="e859f148cff679694f2841d999c6af94aeea17787712660d4273bd00cec8921e" Jul 28 17:59:31.533015 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:31.532932 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e859f148cff679694f2841d999c6af94aeea17787712660d4273bd00cec8921e"} err="failed to get container status \"e859f148cff679694f2841d999c6af94aeea17787712660d4273bd00cec8921e\": rpc error: code = NotFound desc = could not find container \"e859f148cff679694f2841d999c6af94aeea17787712660d4273bd00cec8921e\": container with ID starting with e859f148cff679694f2841d999c6af94aeea17787712660d4273bd00cec8921e not found: ID does not exist" Jul 28 17:59:31.538231 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:31.538211 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6882b4e8-cb9e-46ea-8b21-bb47cb2ffb8c-plugins-config-volume\") pod \"6882b4e8-cb9e-46ea-8b21-bb47cb2ffb8c\" (UID: \"6882b4e8-cb9e-46ea-8b21-bb47cb2ffb8c\") " Jul 28 17:59:31.538355 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:31.538332 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8tjkz\" (UniqueName: \"kubernetes.io/projected/6882b4e8-cb9e-46ea-8b21-bb47cb2ffb8c-kube-api-access-8tjkz\") pod \"6882b4e8-cb9e-46ea-8b21-bb47cb2ffb8c\" (UID: \"6882b4e8-cb9e-46ea-8b21-bb47cb2ffb8c\") " Jul 28 17:59:31.538643 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:31.538617 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6882b4e8-cb9e-46ea-8b21-bb47cb2ffb8c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6882b4e8-cb9e-46ea-8b21-bb47cb2ffb8c" (UID: "6882b4e8-cb9e-46ea-8b21-bb47cb2ffb8c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:59:31.540364 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:31.540337 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6882b4e8-cb9e-46ea-8b21-bb47cb2ffb8c-kube-api-access-8tjkz" (OuterVolumeSpecName: "kube-api-access-8tjkz") pod "6882b4e8-cb9e-46ea-8b21-bb47cb2ffb8c" (UID: "6882b4e8-cb9e-46ea-8b21-bb47cb2ffb8c"). InnerVolumeSpecName "kube-api-access-8tjkz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:59:31.639121 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:31.639073 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8tjkz\" (UniqueName: \"kubernetes.io/projected/6882b4e8-cb9e-46ea-8b21-bb47cb2ffb8c-kube-api-access-8tjkz\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:59:31.639121 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:31.639097 2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6882b4e8-cb9e-46ea-8b21-bb47cb2ffb8c-plugins-config-volume\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:59:31.848505 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:31.848473 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-7fdc90-855bd8bcc4-jvpt7"] Jul 28 17:59:31.854241 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:31.854218 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-7fdc90-855bd8bcc4-jvpt7"] Jul 28 17:59:32.128681 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:32.128637 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6882b4e8-cb9e-46ea-8b21-bb47cb2ffb8c" path="/var/lib/kubelet/pods/6882b4e8-cb9e-46ea-8b21-bb47cb2ffb8c/volumes" Jul 28 17:59:43.357674 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:43.357635 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-20ce8c-hvzzl"] Jul 28 17:59:43.358155 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:43.358069 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6882b4e8-cb9e-46ea-8b21-bb47cb2ffb8c" containerName="payload-processing" Jul 28 17:59:43.358155 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:43.358080 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="6882b4e8-cb9e-46ea-8b21-bb47cb2ffb8c" containerName="payload-processing" Jul 28 17:59:43.358155 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:43.358153 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="6882b4e8-cb9e-46ea-8b21-bb47cb2ffb8c" containerName="payload-processing" Jul 28 17:59:43.361208 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:43.361192 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-20ce8c-hvzzl" Jul 28 17:59:43.363720 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:43.363696 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tl4gw\"" Jul 28 17:59:43.371742 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:43.371703 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-20ce8c-hvzzl"] Jul 28 17:59:43.553400 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:43.553364 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78fcc94b-4e61-4df0-bebf-c4a5c628924e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-20ce8c-hvzzl\" (UID: \"78fcc94b-4e61-4df0-bebf-c4a5c628924e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-20ce8c-hvzzl" Jul 28 17:59:43.653948 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:43.653910 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78fcc94b-4e61-4df0-bebf-c4a5c628924e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-20ce8c-hvzzl\" (UID: \"78fcc94b-4e61-4df0-bebf-c4a5c628924e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-20ce8c-hvzzl" Jul 28 17:59:43.654610 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:43.654584 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78fcc94b-4e61-4df0-bebf-c4a5c628924e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-20ce8c-hvzzl\" (UID: \"78fcc94b-4e61-4df0-bebf-c4a5c628924e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-20ce8c-hvzzl" Jul 28 17:59:43.673142 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:43.673086 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-20ce8c-hvzzl" Jul 28 17:59:44.029931 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:44.029838 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-20ce8c-hvzzl"] Jul 28 17:59:44.033140 ip-10-0-132-11 kubenswrapper[2570]: W0728 17:59:44.033093 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78fcc94b_4e61_4df0_bebf_c4a5c628924e.slice/crio-ae8cfb1c503092e5213e765e37446091e0eb5a10ff4104e9d49c282b3095bcd3 WatchSource:0}: Error finding container ae8cfb1c503092e5213e765e37446091e0eb5a10ff4104e9d49c282b3095bcd3: Status 404 returned error can't find the container with id ae8cfb1c503092e5213e765e37446091e0eb5a10ff4104e9d49c282b3095bcd3 Jul 28 17:59:44.603568 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:44.603470 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-20ce8c-hvzzl" event={"ID":"78fcc94b-4e61-4df0-bebf-c4a5c628924e","Type":"ContainerStarted","Data":"a4cdd43b33587439f47ff249c008ed86ab0c5f984aa1212101395a3bac1590b4"} Jul 28 17:59:44.603568 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:44.603533 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-20ce8c-hvzzl" event={"ID":"78fcc94b-4e61-4df0-bebf-c4a5c628924e","Type":"ContainerStarted","Data":"ae8cfb1c503092e5213e765e37446091e0eb5a10ff4104e9d49c282b3095bcd3"} Jul 28 17:59:44.622247 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:44.622198 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-20ce8c-hvzzl" podStartSLOduration=1.622183127 podStartE2EDuration="1.622183127s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:59:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:59:44.621177783 +0000 UTC m=+3083.230109154" watchObservedRunningTime="2026-07-28 17:59:44.622183127 +0000 UTC m=+3083.231114496" Jul 28 17:59:45.611071 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:45.611033 2570 generic.go:356] "Generic (PLEG): container finished" podID="78fcc94b-4e61-4df0-bebf-c4a5c628924e" containerID="a4cdd43b33587439f47ff249c008ed86ab0c5f984aa1212101395a3bac1590b4" exitCode=0 Jul 28 17:59:45.611476 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:45.611143 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-20ce8c-hvzzl" event={"ID":"78fcc94b-4e61-4df0-bebf-c4a5c628924e","Type":"ContainerDied","Data":"a4cdd43b33587439f47ff249c008ed86ab0c5f984aa1212101395a3bac1590b4"} Jul 28 17:59:46.760587 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:46.760562 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-20ce8c-hvzzl" Jul 28 17:59:46.782910 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:46.782882 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78fcc94b-4e61-4df0-bebf-c4a5c628924e-maas-api-service-ca\") pod \"78fcc94b-4e61-4df0-bebf-c4a5c628924e\" (UID: \"78fcc94b-4e61-4df0-bebf-c4a5c628924e\") " Jul 28 17:59:46.783412 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:46.783365 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78fcc94b-4e61-4df0-bebf-c4a5c628924e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "78fcc94b-4e61-4df0-bebf-c4a5c628924e" (UID: "78fcc94b-4e61-4df0-bebf-c4a5c628924e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:59:46.884294 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:46.884253 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78fcc94b-4e61-4df0-bebf-c4a5c628924e-maas-api-service-ca\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 17:59:47.624079 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:47.624050 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-20ce8c-hvzzl" Jul 28 17:59:47.624248 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:47.624055 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-20ce8c-hvzzl" event={"ID":"78fcc94b-4e61-4df0-bebf-c4a5c628924e","Type":"ContainerDied","Data":"ae8cfb1c503092e5213e765e37446091e0eb5a10ff4104e9d49c282b3095bcd3"} Jul 28 17:59:47.624248 ip-10-0-132-11 kubenswrapper[2570]: I0728 17:59:47.624177 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae8cfb1c503092e5213e765e37446091e0eb5a10ff4104e9d49c282b3095bcd3" Jul 28 18:00:03.456816 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:00:03.456777 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-20ce8c-hvzzl"] Jul 28 18:00:03.462980 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:00:03.462930 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-20ce8c-hvzzl"] Jul 28 18:00:04.120098 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:00:04.120065 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78fcc94b-4e61-4df0-bebf-c4a5c628924e" path="/var/lib/kubelet/pods/78fcc94b-4e61-4df0-bebf-c4a5c628924e/volumes" Jul 28 18:01:17.845319 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:17.845284 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt"] Jul 28 18:01:17.845777 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:17.845513 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" podUID="5a5389c9-9f5d-4cbf-a61a-26c1871bfbca" containerName="istio-proxy" containerID="cri-o://5e3a7d56012b8846ec8ab531388afc0db150a2b240533e146fc993385f699540" gracePeriod=30 Jul 28 18:01:23.109535 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.109472 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" Jul 28 18:01:23.157028 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.156990 2570 generic.go:356] "Generic (PLEG): container finished" podID="5a5389c9-9f5d-4cbf-a61a-26c1871bfbca" containerID="5e3a7d56012b8846ec8ab531388afc0db150a2b240533e146fc993385f699540" exitCode=0 Jul 28 18:01:23.157225 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.157037 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" Jul 28 18:01:23.157225 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.157070 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" event={"ID":"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca","Type":"ContainerDied","Data":"5e3a7d56012b8846ec8ab531388afc0db150a2b240533e146fc993385f699540"} Jul 28 18:01:23.157225 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.157151 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt" event={"ID":"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca","Type":"ContainerDied","Data":"261ffb54cc683c382440870475555a5db20d7981425eb7ee10c697e3f7db83cb"} Jul 28 18:01:23.157225 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.157181 2570 scope.go:117] "RemoveContainer" containerID="5e3a7d56012b8846ec8ab531388afc0db150a2b240533e146fc993385f699540" Jul 28 18:01:23.169238 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.169214 2570 scope.go:117] "RemoveContainer" containerID="95b577752a53116f83271f060fce34748f3ba111d184d7a0239ce282606b0c5d" Jul 28 18:01:23.178018 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.177987 2570 scope.go:117] "RemoveContainer" containerID="5e3a7d56012b8846ec8ab531388afc0db150a2b240533e146fc993385f699540" Jul 28 18:01:23.178306 ip-10-0-132-11 kubenswrapper[2570]: E0728 18:01:23.178289 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e3a7d56012b8846ec8ab531388afc0db150a2b240533e146fc993385f699540\": container with ID starting with 5e3a7d56012b8846ec8ab531388afc0db150a2b240533e146fc993385f699540 not found: ID does not exist" containerID="5e3a7d56012b8846ec8ab531388afc0db150a2b240533e146fc993385f699540" Jul 28 18:01:23.178370 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.178315 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e3a7d56012b8846ec8ab531388afc0db150a2b240533e146fc993385f699540"} err="failed to get container status \"5e3a7d56012b8846ec8ab531388afc0db150a2b240533e146fc993385f699540\": rpc error: code = NotFound desc = could not find container \"5e3a7d56012b8846ec8ab531388afc0db150a2b240533e146fc993385f699540\": container with ID starting with 5e3a7d56012b8846ec8ab531388afc0db150a2b240533e146fc993385f699540 not found: ID does not exist" Jul 28 18:01:23.178370 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.178333 2570 scope.go:117] "RemoveContainer" containerID="95b577752a53116f83271f060fce34748f3ba111d184d7a0239ce282606b0c5d" Jul 28 18:01:23.178608 ip-10-0-132-11 kubenswrapper[2570]: E0728 18:01:23.178581 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95b577752a53116f83271f060fce34748f3ba111d184d7a0239ce282606b0c5d\": container with ID starting with 95b577752a53116f83271f060fce34748f3ba111d184d7a0239ce282606b0c5d not found: ID does not exist" containerID="95b577752a53116f83271f060fce34748f3ba111d184d7a0239ce282606b0c5d" Jul 28 18:01:23.178656 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.178614 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95b577752a53116f83271f060fce34748f3ba111d184d7a0239ce282606b0c5d"} err="failed to get container status \"95b577752a53116f83271f060fce34748f3ba111d184d7a0239ce282606b0c5d\": rpc error: code = NotFound desc = could not find container \"95b577752a53116f83271f060fce34748f3ba111d184d7a0239ce282606b0c5d\": container with ID starting with 95b577752a53116f83271f060fce34748f3ba111d184d7a0239ce282606b0c5d not found: ID does not exist" Jul 28 18:01:23.251973 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.251944 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-istiod-ca-cert\") pod \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\" (UID: \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\") " Jul 28 18:01:23.252147 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.252002 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-istio-envoy\") pod \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\" (UID: \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\") " Jul 28 18:01:23.252147 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.252032 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-istio-data\") pod \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\" (UID: \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\") " Jul 28 18:01:23.252147 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.252075 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-workload-certs\") pod \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\" (UID: \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\") " Jul 28 18:01:23.252258 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.252173 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-workload-socket\") pod \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\" (UID: \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\") " Jul 28 18:01:23.252258 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.252224 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-istio-token\") pod \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\" (UID: \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\") " Jul 28 18:01:23.252353 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.252255 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-istio-podinfo\") pod \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\" (UID: \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\") " Jul 28 18:01:23.252353 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.252299 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-credential-socket\") pod \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\" (UID: \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\") " Jul 28 18:01:23.252353 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.252335 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wf57x\" (UniqueName: \"kubernetes.io/projected/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-kube-api-access-wf57x\") pod \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\" (UID: \"5a5389c9-9f5d-4cbf-a61a-26c1871bfbca\") " Jul 28 18:01:23.252515 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.252416 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5a5389c9-9f5d-4cbf-a61a-26c1871bfbca" (UID: "5a5389c9-9f5d-4cbf-a61a-26c1871bfbca"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 18:01:23.252579 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.252505 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5a5389c9-9f5d-4cbf-a61a-26c1871bfbca" (UID: "5a5389c9-9f5d-4cbf-a61a-26c1871bfbca"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 18:01:23.252745 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.252721 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5a5389c9-9f5d-4cbf-a61a-26c1871bfbca" (UID: "5a5389c9-9f5d-4cbf-a61a-26c1871bfbca"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 18:01:23.252841 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.252763 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5a5389c9-9f5d-4cbf-a61a-26c1871bfbca" (UID: "5a5389c9-9f5d-4cbf-a61a-26c1871bfbca"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 18:01:23.252918 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.252895 2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-workload-socket\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 18:01:23.252976 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.252920 2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-credential-socket\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 18:01:23.252976 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.252935 2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-istiod-ca-cert\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 18:01:23.252976 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.252951 2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-workload-certs\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 18:01:23.252976 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.252930 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-istio-data" (OuterVolumeSpecName: "istio-data") pod "5a5389c9-9f5d-4cbf-a61a-26c1871bfbca" (UID: "5a5389c9-9f5d-4cbf-a61a-26c1871bfbca"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 18:01:23.254660 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.254633 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5a5389c9-9f5d-4cbf-a61a-26c1871bfbca" (UID: "5a5389c9-9f5d-4cbf-a61a-26c1871bfbca"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 18:01:23.254763 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.254657 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-kube-api-access-wf57x" (OuterVolumeSpecName: "kube-api-access-wf57x") pod "5a5389c9-9f5d-4cbf-a61a-26c1871bfbca" (UID: "5a5389c9-9f5d-4cbf-a61a-26c1871bfbca"). InnerVolumeSpecName "kube-api-access-wf57x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 18:01:23.254763 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.254703 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-istio-token" (OuterVolumeSpecName: "istio-token") pod "5a5389c9-9f5d-4cbf-a61a-26c1871bfbca" (UID: "5a5389c9-9f5d-4cbf-a61a-26c1871bfbca"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 18:01:23.255159 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.255134 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5a5389c9-9f5d-4cbf-a61a-26c1871bfbca" (UID: "5a5389c9-9f5d-4cbf-a61a-26c1871bfbca"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 28 18:01:23.353641 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.353606 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wf57x\" (UniqueName: \"kubernetes.io/projected/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-kube-api-access-wf57x\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 18:01:23.353641 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.353637 2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-istio-envoy\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 18:01:23.353641 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.353648 2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-istio-data\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 18:01:23.353880 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.353656 2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-istio-token\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 18:01:23.353880 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.353663 2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca-istio-podinfo\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 18:01:23.490549 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.490515 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt"] Jul 28 18:01:23.501100 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:23.501075 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-857ec3-openshift-default-64cdfb6998-l4stt"] Jul 28 18:01:24.122174 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:01:24.122140 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a5389c9-9f5d-4cbf-a61a-26c1871bfbca" path="/var/lib/kubelet/pods/5a5389c9-9f5d-4cbf-a61a-26c1871bfbca/volumes" Jul 28 18:02:05.951916 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:05.951883 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx"] Jul 28 18:02:05.952495 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:05.952088 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" podUID="38532552-35c9-4e91-a557-9a176b2d7f22" containerName="istio-proxy" containerID="cri-o://f30b06a787b989f7882ddff697dbbf3dd4970e9354bfc32e00460c01563452a4" gracePeriod=30 Jul 28 18:02:06.671420 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:06.671387 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-84b4d8cb66-bnw9p_4b8835fa-ff72-4eb8-943e-fe808e69690b/authorino/0.log" Jul 28 18:02:11.218837 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.218773 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" Jul 28 18:02:11.319331 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.319299 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38532552-35c9-4e91-a557-9a176b2d7f22-workload-socket\") pod \"38532552-35c9-4e91-a557-9a176b2d7f22\" (UID: \"38532552-35c9-4e91-a557-9a176b2d7f22\") " Jul 28 18:02:11.319331 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.319339 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38532552-35c9-4e91-a557-9a176b2d7f22-istiod-ca-cert\") pod \"38532552-35c9-4e91-a557-9a176b2d7f22\" (UID: \"38532552-35c9-4e91-a557-9a176b2d7f22\") " Jul 28 18:02:11.319565 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.319360 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38532552-35c9-4e91-a557-9a176b2d7f22-istio-token\") pod \"38532552-35c9-4e91-a557-9a176b2d7f22\" (UID: \"38532552-35c9-4e91-a557-9a176b2d7f22\") " Jul 28 18:02:11.319565 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.319377 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38532552-35c9-4e91-a557-9a176b2d7f22-istio-data\") pod \"38532552-35c9-4e91-a557-9a176b2d7f22\" (UID: \"38532552-35c9-4e91-a557-9a176b2d7f22\") " Jul 28 18:02:11.319565 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.319421 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38532552-35c9-4e91-a557-9a176b2d7f22-istio-envoy\") pod \"38532552-35c9-4e91-a557-9a176b2d7f22\" (UID: \"38532552-35c9-4e91-a557-9a176b2d7f22\") " Jul 28 18:02:11.319565 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.319437 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mn5hw\" (UniqueName: \"kubernetes.io/projected/38532552-35c9-4e91-a557-9a176b2d7f22-kube-api-access-mn5hw\") pod \"38532552-35c9-4e91-a557-9a176b2d7f22\" (UID: \"38532552-35c9-4e91-a557-9a176b2d7f22\") " Jul 28 18:02:11.319565 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.319482 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38532552-35c9-4e91-a557-9a176b2d7f22-workload-certs\") pod \"38532552-35c9-4e91-a557-9a176b2d7f22\" (UID: \"38532552-35c9-4e91-a557-9a176b2d7f22\") " Jul 28 18:02:11.319565 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.319513 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38532552-35c9-4e91-a557-9a176b2d7f22-credential-socket\") pod \"38532552-35c9-4e91-a557-9a176b2d7f22\" (UID: \"38532552-35c9-4e91-a557-9a176b2d7f22\") " Jul 28 18:02:11.319565 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.319566 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38532552-35c9-4e91-a557-9a176b2d7f22-istio-podinfo\") pod \"38532552-35c9-4e91-a557-9a176b2d7f22\" (UID: \"38532552-35c9-4e91-a557-9a176b2d7f22\") " Jul 28 18:02:11.319924 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.319647 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38532552-35c9-4e91-a557-9a176b2d7f22-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "38532552-35c9-4e91-a557-9a176b2d7f22" (UID: "38532552-35c9-4e91-a557-9a176b2d7f22"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 18:02:11.319924 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.319802 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38532552-35c9-4e91-a557-9a176b2d7f22-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "38532552-35c9-4e91-a557-9a176b2d7f22" (UID: "38532552-35c9-4e91-a557-9a176b2d7f22"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 18:02:11.319924 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.319814 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38532552-35c9-4e91-a557-9a176b2d7f22-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "38532552-35c9-4e91-a557-9a176b2d7f22" (UID: "38532552-35c9-4e91-a557-9a176b2d7f22"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 18:02:11.320075 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.319947 2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38532552-35c9-4e91-a557-9a176b2d7f22-workload-socket\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 18:02:11.320075 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.319966 2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38532552-35c9-4e91-a557-9a176b2d7f22-istiod-ca-cert\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 18:02:11.320075 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.319981 2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38532552-35c9-4e91-a557-9a176b2d7f22-workload-certs\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 18:02:11.320265 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.320089 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38532552-35c9-4e91-a557-9a176b2d7f22-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "38532552-35c9-4e91-a557-9a176b2d7f22" (UID: "38532552-35c9-4e91-a557-9a176b2d7f22"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 18:02:11.320439 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.320412 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38532552-35c9-4e91-a557-9a176b2d7f22-istio-data" (OuterVolumeSpecName: "istio-data") pod "38532552-35c9-4e91-a557-9a176b2d7f22" (UID: "38532552-35c9-4e91-a557-9a176b2d7f22"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 18:02:11.321774 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.321747 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38532552-35c9-4e91-a557-9a176b2d7f22-istio-token" (OuterVolumeSpecName: "istio-token") pod "38532552-35c9-4e91-a557-9a176b2d7f22" (UID: "38532552-35c9-4e91-a557-9a176b2d7f22"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 18:02:11.321927 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.321909 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/38532552-35c9-4e91-a557-9a176b2d7f22-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "38532552-35c9-4e91-a557-9a176b2d7f22" (UID: "38532552-35c9-4e91-a557-9a176b2d7f22"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 28 18:02:11.322132 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.322086 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38532552-35c9-4e91-a557-9a176b2d7f22-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "38532552-35c9-4e91-a557-9a176b2d7f22" (UID: "38532552-35c9-4e91-a557-9a176b2d7f22"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 18:02:11.322192 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.322142 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38532552-35c9-4e91-a557-9a176b2d7f22-kube-api-access-mn5hw" (OuterVolumeSpecName: "kube-api-access-mn5hw") pod "38532552-35c9-4e91-a557-9a176b2d7f22" (UID: "38532552-35c9-4e91-a557-9a176b2d7f22"). InnerVolumeSpecName "kube-api-access-mn5hw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 18:02:11.420827 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.420793 2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38532552-35c9-4e91-a557-9a176b2d7f22-credential-socket\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 18:02:11.420827 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.420823 2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38532552-35c9-4e91-a557-9a176b2d7f22-istio-podinfo\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 18:02:11.421017 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.420836 2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38532552-35c9-4e91-a557-9a176b2d7f22-istio-token\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 18:02:11.421017 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.420845 2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38532552-35c9-4e91-a557-9a176b2d7f22-istio-data\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 18:02:11.421017 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.420853 2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38532552-35c9-4e91-a557-9a176b2d7f22-istio-envoy\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 18:02:11.421017 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.420861 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mn5hw\" (UniqueName: \"kubernetes.io/projected/38532552-35c9-4e91-a557-9a176b2d7f22-kube-api-access-mn5hw\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\"" Jul 28 18:02:11.425596 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.425564 2570 generic.go:356] "Generic (PLEG): container finished" podID="38532552-35c9-4e91-a557-9a176b2d7f22" containerID="f30b06a787b989f7882ddff697dbbf3dd4970e9354bfc32e00460c01563452a4" exitCode=0 Jul 28 18:02:11.425709 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.425605 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" event={"ID":"38532552-35c9-4e91-a557-9a176b2d7f22","Type":"ContainerDied","Data":"f30b06a787b989f7882ddff697dbbf3dd4970e9354bfc32e00460c01563452a4"} Jul 28 18:02:11.425709 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.425614 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" Jul 28 18:02:11.425709 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.425648 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx" event={"ID":"38532552-35c9-4e91-a557-9a176b2d7f22","Type":"ContainerDied","Data":"73222d32c66e5100ec0ae20874ac79bb4e087b7d9360b81370d2b29db5c4e6e0"} Jul 28 18:02:11.425709 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.425674 2570 scope.go:117] "RemoveContainer" containerID="f30b06a787b989f7882ddff697dbbf3dd4970e9354bfc32e00460c01563452a4" Jul 28 18:02:11.435296 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.435274 2570 scope.go:117] "RemoveContainer" containerID="83810d570b2f54f04322e5ed1ec155a54a4563c6ebb44310ae2f137dbcc74e39" Jul 28 18:02:11.443528 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.443513 2570 scope.go:117] "RemoveContainer" containerID="f30b06a787b989f7882ddff697dbbf3dd4970e9354bfc32e00460c01563452a4" Jul 28 18:02:11.443771 ip-10-0-132-11 kubenswrapper[2570]: E0728 18:02:11.443751 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f30b06a787b989f7882ddff697dbbf3dd4970e9354bfc32e00460c01563452a4\": container with ID starting with f30b06a787b989f7882ddff697dbbf3dd4970e9354bfc32e00460c01563452a4 not found: ID does not exist" containerID="f30b06a787b989f7882ddff697dbbf3dd4970e9354bfc32e00460c01563452a4" Jul 28 18:02:11.443810 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.443779 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f30b06a787b989f7882ddff697dbbf3dd4970e9354bfc32e00460c01563452a4"} err="failed to get container status \"f30b06a787b989f7882ddff697dbbf3dd4970e9354bfc32e00460c01563452a4\": rpc error: code = NotFound desc = could not find container \"f30b06a787b989f7882ddff697dbbf3dd4970e9354bfc32e00460c01563452a4\": container with ID starting with f30b06a787b989f7882ddff697dbbf3dd4970e9354bfc32e00460c01563452a4 not found: ID does not exist" Jul 28 18:02:11.443810 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.443794 2570 scope.go:117] "RemoveContainer" containerID="83810d570b2f54f04322e5ed1ec155a54a4563c6ebb44310ae2f137dbcc74e39" Jul 28 18:02:11.444003 ip-10-0-132-11 kubenswrapper[2570]: E0728 18:02:11.443989 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83810d570b2f54f04322e5ed1ec155a54a4563c6ebb44310ae2f137dbcc74e39\": container with ID starting with 83810d570b2f54f04322e5ed1ec155a54a4563c6ebb44310ae2f137dbcc74e39 not found: ID does not exist" containerID="83810d570b2f54f04322e5ed1ec155a54a4563c6ebb44310ae2f137dbcc74e39" Jul 28 18:02:11.444044 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.444007 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83810d570b2f54f04322e5ed1ec155a54a4563c6ebb44310ae2f137dbcc74e39"} err="failed to get container status \"83810d570b2f54f04322e5ed1ec155a54a4563c6ebb44310ae2f137dbcc74e39\": rpc error: code = NotFound desc = could not find container \"83810d570b2f54f04322e5ed1ec155a54a4563c6ebb44310ae2f137dbcc74e39\": container with ID starting with 83810d570b2f54f04322e5ed1ec155a54a4563c6ebb44310ae2f137dbcc74e39 not found: ID does not exist" Jul 28 18:02:11.450778 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.450756 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx"] Jul 28 18:02:11.455173 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:11.455152 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-f24b7c-openshift-default-68fffc9b8-zt6bx"] Jul 28 18:02:12.121208 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:12.121177 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38532552-35c9-4e91-a557-9a176b2d7f22" path="/var/lib/kubelet/pods/38532552-35c9-4e91-a557-9a176b2d7f22/volumes" Jul 28 18:02:12.407206 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:12.407103 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-s9lmn_e988c6ed-5788-4f9c-91d3-263b35ab7d49/manager/0.log" Jul 28 18:02:12.527357 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:12.527331 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-lmn47_039b79ed-2f6b-41ba-8529-fe1a6be83441/manager/0.log" Jul 28 18:02:12.659310 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:12.659236 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7d6f646b66-jww2t_4facf865-9753-4107-82ef-fd66a0bb9e93/manager/0.log" Jul 28 18:02:12.776555 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:12.776526 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-7t86x_9eb4c97f-be9f-4584-b563-9a7e7efc1d2a/server/0.log" Jul 28 18:02:12.900710 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:12.900683 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-mpxpz_650f3a82-4de5-4bd2-ba3b-4f9c4c666f53/manager/0.log" Jul 28 18:02:13.207897 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:13.207872 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d8c7b7dcd-lc72m_4c258043-1b2d-4bd2-abfb-c46a681d788b/copy-manifests/0.log" Jul 28 18:02:13.216679 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:13.216658 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d8c7b7dcd-lc72m_4c258043-1b2d-4bd2-abfb-c46a681d788b/manager/0.log" Jul 28 18:02:13.706885 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:13.706850 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-76576b46b5-n4hkm_e11a57c0-9013-4492-9722-fd0aa4bb2c92/maas-api/0.log" Jul 28 18:02:13.951509 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:13.951484 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29754345-8pgwg_dd167ac2-c609-4a60-9bc1-4c26d25de67c/cleanup/0.log" Jul 28 18:02:14.189816 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:14.189791 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-9kxc6_c7a628d7-7c96-4a8a-87f7-abb463d1dfa8/postgres/0.log" Jul 28 18:02:15.555658 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:15.555625 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-84b4d8cb66-bnw9p_4b8835fa-ff72-4eb8-943e-fe808e69690b/authorino/0.log" Jul 28 18:02:15.683647 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:15.683615 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-wvxrn_b5723932-0f49-4854-9a46-8ab1f3fdec8d/manager/0.log" Jul 28 18:02:15.805387 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:15.805359 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-4z5f5_776d3861-f435-4f91-b0d6-b4d3a1124820/manager/0.log" Jul 28 18:02:16.049244 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:16.049219 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-r85k2_33717d5a-d5aa-4b97-bcec-a56118a8afee/registry-server/0.log" Jul 28 18:02:16.179459 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:16.179435 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-ds7ns_27bc78bb-e7d4-4796-9d00-7c9454975d57/manager/0.log" Jul 28 18:02:16.822421 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:16.822394 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7_9bbda9ba-84f0-40a5-979d-9ea282384e4e/istio-proxy/0.log" Jul 28 18:02:17.203238 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:17.203206 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77b678cfbc-gg5gz_1d224fae-cf17-4bdb-baf5-1dde1cd43871/kube-auth-proxy/0.log" Jul 28 18:02:17.333544 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:17.333514 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-vnfxw_ecd3cc49-6ce7-4db3-a0d0-797b1600822b/istio-proxy/1.log" Jul 28 18:02:17.453786 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:17.453711 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-4n9g5_3c4430fa-54ac-4a06-87fc-08555d52774b/payload-pre-processing/0.log" Jul 28 18:02:17.709324 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:17.709243 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64c477cb5c-9stkr_af095ddd-e1cd-4148-9796-d60be4fef0e2/router/0.log" Jul 28 18:02:18.222562 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:18.222534 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-k47ks_fae7d027-f7a8-494a-b88f-316046e21f1b/main/0.log" Jul 28 18:02:18.461350 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:18.461323 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkqm67_958be73c-2e93-409e-92f0-e4013e60c958/main/0.log" Jul 28 18:02:18.712613 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:18.712586 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-h4rgl_a4c06325-fc6a-473a-812a-b6f72bd4dfc9/main/0.log" Jul 28 18:02:27.375084 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:27.375053 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-fbzsz_9baa058e-a765-4c65-b5a6-ce0a7c022d97/global-pull-secret-syncer/0.log" Jul 28 18:02:27.522710 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:27.522674 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-dmc56_e2d09d1f-c076-4633-aa92-acb1adc2cbfe/konnectivity-agent/0.log" Jul 28 18:02:27.618835 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:27.618808 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-11.ec2.internal_a8cc528b9c87d3a6a58c64430cacff3d/haproxy/0.log" Jul 28 18:02:32.706523 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:32.706490 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-84b4d8cb66-bnw9p_4b8835fa-ff72-4eb8-943e-fe808e69690b/authorino/0.log" Jul 28 18:02:32.767447 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:32.767420 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-wvxrn_b5723932-0f49-4854-9a46-8ab1f3fdec8d/manager/0.log" Jul 28 18:02:32.832944 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:32.832917 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-4z5f5_776d3861-f435-4f91-b0d6-b4d3a1124820/manager/0.log" Jul 28 18:02:32.918534 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:32.918503 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-r85k2_33717d5a-d5aa-4b97-bcec-a56118a8afee/registry-server/0.log" Jul 28 18:02:33.118677 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:33.118583 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-ds7ns_27bc78bb-e7d4-4796-9d00-7c9454975d57/manager/0.log" Jul 28 18:02:35.343358 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:35.343295 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-m4bxm_c6ad5592-e8fc-41e7-8fcf-caa57c3eae35/cluster-monitoring-operator/0.log" Jul 28 18:02:35.624821 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:35.624793 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q7fs6_f351cfd4-5d9a-4977-99fc-c37be3098628/node-exporter/0.log" Jul 28 18:02:35.650540 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:35.650513 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q7fs6_f351cfd4-5d9a-4977-99fc-c37be3098628/kube-rbac-proxy/0.log" Jul 28 18:02:35.674160 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:35.674139 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q7fs6_f351cfd4-5d9a-4977-99fc-c37be3098628/init-textfile/0.log" Jul 28 18:02:37.264268 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:37.264234 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-f5qzw/perf-node-gather-daemonset-vw69b"] Jul 28 18:02:37.264772 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:37.264651 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38532552-35c9-4e91-a557-9a176b2d7f22" containerName="istio-proxy" Jul 28 18:02:37.264772 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:37.264662 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="38532552-35c9-4e91-a557-9a176b2d7f22" containerName="istio-proxy" Jul 28 18:02:37.264772 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:37.264688 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a5389c9-9f5d-4cbf-a61a-26c1871bfbca" containerName="istio-proxy" Jul 28 18:02:37.264772 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:37.264694 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a5389c9-9f5d-4cbf-a61a-26c1871bfbca" containerName="istio-proxy" Jul 28 18:02:37.264772 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:37.264701 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78fcc94b-4e61-4df0-bebf-c4a5c628924e" containerName="revoke-keys" Jul 28 18:02:37.264772 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:37.264707 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="78fcc94b-4e61-4df0-bebf-c4a5c628924e" containerName="revoke-keys" Jul 28 18:02:37.264772 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:37.264714 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38532552-35c9-4e91-a557-9a176b2d7f22" containerName="istio-proxy" Jul 28 18:02:37.264772 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:37.264719 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="38532552-35c9-4e91-a557-9a176b2d7f22" containerName="istio-proxy" Jul 28 18:02:37.264772 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:37.264742 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a5389c9-9f5d-4cbf-a61a-26c1871bfbca" containerName="istio-proxy" Jul 28 18:02:37.264772 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:37.264750 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a5389c9-9f5d-4cbf-a61a-26c1871bfbca" containerName="istio-proxy" Jul 28 18:02:37.265161 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:37.264838 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a5389c9-9f5d-4cbf-a61a-26c1871bfbca" containerName="istio-proxy" Jul 28 18:02:37.265161 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:37.264849 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="78fcc94b-4e61-4df0-bebf-c4a5c628924e" containerName="revoke-keys" Jul 28 18:02:37.265161 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:37.264859 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="38532552-35c9-4e91-a557-9a176b2d7f22" containerName="istio-proxy" Jul 28 18:02:37.265161 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:37.264866 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="38532552-35c9-4e91-a557-9a176b2d7f22" containerName="istio-proxy" Jul 28 18:02:37.265161 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:37.264875 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a5389c9-9f5d-4cbf-a61a-26c1871bfbca" containerName="istio-proxy" Jul 28 18:02:37.267656 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:37.267639 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f5qzw/perf-node-gather-daemonset-vw69b" Jul 28 18:02:37.270371 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:37.270347 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-f5qzw\"/\"kube-root-ca.crt\"" Jul 28 18:02:37.271403 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:37.271387 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-f5qzw\"/\"default-dockercfg-rmzjs\"" Jul 28 18:02:37.271504 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:37.271440 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-f5qzw\"/\"openshift-service-ca.crt\"" Jul 28 18:02:37.276289 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:37.276261 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-f5qzw/perf-node-gather-daemonset-vw69b"] Jul 28 18:02:37.358347 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:37.358311 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3ed573d5-19a6-4b79-baa3-eab8d9a2cb02-proc\") pod \"perf-node-gather-daemonset-vw69b\" (UID: \"3ed573d5-19a6-4b79-baa3-eab8d9a2cb02\") " pod="openshift-must-gather-f5qzw/perf-node-gather-daemonset-vw69b" Jul 28 18:02:37.358539 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:37.358355 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3ed573d5-19a6-4b79-baa3-eab8d9a2cb02-sys\") pod \"perf-node-gather-daemonset-vw69b\" (UID: \"3ed573d5-19a6-4b79-baa3-eab8d9a2cb02\") " pod="openshift-must-gather-f5qzw/perf-node-gather-daemonset-vw69b" Jul 28 18:02:37.358539 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:37.358421 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3ed573d5-19a6-4b79-baa3-eab8d9a2cb02-lib-modules\") pod \"perf-node-gather-daemonset-vw69b\" (UID: \"3ed573d5-19a6-4b79-baa3-eab8d9a2cb02\") " pod="openshift-must-gather-f5qzw/perf-node-gather-daemonset-vw69b" Jul 28 18:02:37.358680 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:37.358574 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3ed573d5-19a6-4b79-baa3-eab8d9a2cb02-podres\") pod \"perf-node-gather-daemonset-vw69b\" (UID: \"3ed573d5-19a6-4b79-baa3-eab8d9a2cb02\") " pod="openshift-must-gather-f5qzw/perf-node-gather-daemonset-vw69b" Jul 28 18:02:37.358680 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:37.358633 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xms5p\" (UniqueName: \"kubernetes.io/projected/3ed573d5-19a6-4b79-baa3-eab8d9a2cb02-kube-api-access-xms5p\") pod \"perf-node-gather-daemonset-vw69b\" (UID: \"3ed573d5-19a6-4b79-baa3-eab8d9a2cb02\") " pod="openshift-must-gather-f5qzw/perf-node-gather-daemonset-vw69b" Jul 28 18:02:37.459581 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:37.459549 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3ed573d5-19a6-4b79-baa3-eab8d9a2cb02-lib-modules\") pod \"perf-node-gather-daemonset-vw69b\" (UID: \"3ed573d5-19a6-4b79-baa3-eab8d9a2cb02\") " pod="openshift-must-gather-f5qzw/perf-node-gather-daemonset-vw69b" Jul 28 18:02:37.459760 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:37.459634 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3ed573d5-19a6-4b79-baa3-eab8d9a2cb02-podres\") pod \"perf-node-gather-daemonset-vw69b\" (UID: \"3ed573d5-19a6-4b79-baa3-eab8d9a2cb02\") " pod="openshift-must-gather-f5qzw/perf-node-gather-daemonset-vw69b" Jul 28 18:02:37.459760 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:37.459665 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xms5p\" (UniqueName: \"kubernetes.io/projected/3ed573d5-19a6-4b79-baa3-eab8d9a2cb02-kube-api-access-xms5p\") pod \"perf-node-gather-daemonset-vw69b\" (UID: \"3ed573d5-19a6-4b79-baa3-eab8d9a2cb02\") " pod="openshift-must-gather-f5qzw/perf-node-gather-daemonset-vw69b" Jul 28 18:02:37.459760 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:37.459690 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3ed573d5-19a6-4b79-baa3-eab8d9a2cb02-proc\") pod \"perf-node-gather-daemonset-vw69b\" (UID: \"3ed573d5-19a6-4b79-baa3-eab8d9a2cb02\") " pod="openshift-must-gather-f5qzw/perf-node-gather-daemonset-vw69b" Jul 28 18:02:37.459760 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:37.459705 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3ed573d5-19a6-4b79-baa3-eab8d9a2cb02-sys\") pod \"perf-node-gather-daemonset-vw69b\" (UID: \"3ed573d5-19a6-4b79-baa3-eab8d9a2cb02\") " pod="openshift-must-gather-f5qzw/perf-node-gather-daemonset-vw69b" Jul 28 18:02:37.459760 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:37.459730 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3ed573d5-19a6-4b79-baa3-eab8d9a2cb02-lib-modules\") pod \"perf-node-gather-daemonset-vw69b\" (UID: \"3ed573d5-19a6-4b79-baa3-eab8d9a2cb02\") " pod="openshift-must-gather-f5qzw/perf-node-gather-daemonset-vw69b" Jul 28 18:02:37.460022 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:37.459771 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3ed573d5-19a6-4b79-baa3-eab8d9a2cb02-sys\") pod \"perf-node-gather-daemonset-vw69b\" (UID: \"3ed573d5-19a6-4b79-baa3-eab8d9a2cb02\") " pod="openshift-must-gather-f5qzw/perf-node-gather-daemonset-vw69b" Jul 28 18:02:37.460022 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:37.459789 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3ed573d5-19a6-4b79-baa3-eab8d9a2cb02-proc\") pod \"perf-node-gather-daemonset-vw69b\" (UID: \"3ed573d5-19a6-4b79-baa3-eab8d9a2cb02\") " pod="openshift-must-gather-f5qzw/perf-node-gather-daemonset-vw69b" Jul 28 18:02:37.460022 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:37.459806 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3ed573d5-19a6-4b79-baa3-eab8d9a2cb02-podres\") pod \"perf-node-gather-daemonset-vw69b\" (UID: \"3ed573d5-19a6-4b79-baa3-eab8d9a2cb02\") " pod="openshift-must-gather-f5qzw/perf-node-gather-daemonset-vw69b" Jul 28 18:02:37.469321 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:37.469298 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xms5p\" (UniqueName: \"kubernetes.io/projected/3ed573d5-19a6-4b79-baa3-eab8d9a2cb02-kube-api-access-xms5p\") pod \"perf-node-gather-daemonset-vw69b\" (UID: \"3ed573d5-19a6-4b79-baa3-eab8d9a2cb02\") " pod="openshift-must-gather-f5qzw/perf-node-gather-daemonset-vw69b" Jul 28 18:02:37.578373 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:37.578267 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f5qzw/perf-node-gather-daemonset-vw69b" Jul 28 18:02:37.661203 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:37.661168 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-ngj99_52730019-2aad-4136-91b5-1b82eb52e0d1/networking-console-plugin/0.log" Jul 28 18:02:37.715244 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:37.715218 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-f5qzw/perf-node-gather-daemonset-vw69b"] Jul 28 18:02:37.717190 ip-10-0-132-11 kubenswrapper[2570]: W0728 18:02:37.717155 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3ed573d5_19a6_4b79_baa3_eab8d9a2cb02.slice/crio-9ca0c554c44a8454320fd432252c32915211d1832c374321b1fa6123a719b835 WatchSource:0}: Error finding container 9ca0c554c44a8454320fd432252c32915211d1832c374321b1fa6123a719b835: Status 404 returned error can't find the container with id 9ca0c554c44a8454320fd432252c32915211d1832c374321b1fa6123a719b835 Jul 28 18:02:38.280700 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:38.280669 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-slfqf_ecb82800-8cf0-4ef9-926c-155172fad7ff/console-operator/1.log" Jul 28 18:02:38.285467 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:38.285448 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-slfqf_ecb82800-8cf0-4ef9-926c-155172fad7ff/console-operator/2.log" Jul 28 18:02:38.575836 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:38.575747 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f5qzw/perf-node-gather-daemonset-vw69b" event={"ID":"3ed573d5-19a6-4b79-baa3-eab8d9a2cb02","Type":"ContainerStarted","Data":"0f0234c996e7b885098ca71b9e26399061cb895f7b793917c66eadd6cc9a23a9"} Jul 28 18:02:38.575836 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:38.575800 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f5qzw/perf-node-gather-daemonset-vw69b" event={"ID":"3ed573d5-19a6-4b79-baa3-eab8d9a2cb02","Type":"ContainerStarted","Data":"9ca0c554c44a8454320fd432252c32915211d1832c374321b1fa6123a719b835"} Jul 28 18:02:38.576019 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:38.575840 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-f5qzw/perf-node-gather-daemonset-vw69b" Jul 28 18:02:38.600851 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:38.600805 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-f5qzw/perf-node-gather-daemonset-vw69b" podStartSLOduration=1.600790962 podStartE2EDuration="1.600790962s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 18:02:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:02:38.600056084 +0000 UTC m=+3257.208987458" watchObservedRunningTime="2026-07-28 18:02:38.600790962 +0000 UTC m=+3257.209722356" Jul 28 18:02:39.573391 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:39.573360 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-765pv_ebeb51c6-5d43-493d-b8bd-e56f9993e032/volume-data-source-validator/0.log" Jul 28 18:02:40.796185 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:40.796153 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-49xwx_fd91b2e7-ff77-474f-a69b-47a84fe174a3/dns/0.log" Jul 28 18:02:40.841669 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:40.841640 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-49xwx_fd91b2e7-ff77-474f-a69b-47a84fe174a3/kube-rbac-proxy/0.log" Jul 28 18:02:41.171862 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:41.171837 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zbmdl_dd54c89e-8c35-404b-8804-ca3e45d384b4/dns-node-resolver/0.log" Jul 28 18:02:41.745020 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:41.744988 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7b5446bd95-ngjw4_0aa92363-6609-459f-9ce0-bbc1700a8bd8/registry/0.log" Jul 28 18:02:41.810475 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:41.810444 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cmx6g_56d1a325-976b-4060-b1a4-1a4b6be0bd18/node-ca/0.log" Jul 28 18:02:42.979378 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:42.979345 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7_9bbda9ba-84f0-40a5-979d-9ea282384e4e/istio-proxy/0.log" Jul 28 18:02:43.275010 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:43.274927 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77b678cfbc-gg5gz_1d224fae-cf17-4bdb-baf5-1dde1cd43871/kube-auth-proxy/0.log" Jul 28 18:02:43.318789 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:43.318743 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-vnfxw_ecd3cc49-6ce7-4db3-a0d0-797b1600822b/istio-proxy/0.log" Jul 28 18:02:43.343968 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:43.343932 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-vnfxw_ecd3cc49-6ce7-4db3-a0d0-797b1600822b/istio-proxy/1.log" Jul 28 18:02:43.386701 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:43.386669 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-4n9g5_3c4430fa-54ac-4a06-87fc-08555d52774b/payload-pre-processing/0.log" Jul 28 18:02:43.477919 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:43.477896 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64c477cb5c-9stkr_af095ddd-e1cd-4148-9796-d60be4fef0e2/router/0.log" Jul 28 18:02:44.421952 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:44.421921 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-s9t99_78838e40-d32b-4f26-9e51-3dd3352026c3/serve-healthcheck-canary/0.log" Jul 28 18:02:44.591386 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:44.591362 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-f5qzw/perf-node-gather-daemonset-vw69b" Jul 28 18:02:45.018624 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:45.018587 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-bpkxs_8f9b8ad8-774d-4ab8-9826-94eb5a37743f/insights-operator/0.log" Jul 28 18:02:45.019253 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:45.019236 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-bpkxs_8f9b8ad8-774d-4ab8-9826-94eb5a37743f/insights-operator/1.log" Jul 28 18:02:45.519762 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:45.519733 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n776k_39aa1164-b895-45e9-b64a-5220376fdd95/kube-rbac-proxy/0.log" Jul 28 18:02:45.566281 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:45.566249 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n776k_39aa1164-b895-45e9-b64a-5220376fdd95/exporter/0.log" Jul 28 18:02:45.621066 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:45.621027 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n776k_39aa1164-b895-45e9-b64a-5220376fdd95/extractor/0.log" Jul 28 18:02:48.010965 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:48.010881 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7_9bbda9ba-84f0-40a5-979d-9ea282384e4e/istio-proxy/0.log" Jul 28 18:02:48.250296 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:48.250268 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77b678cfbc-gg5gz_1d224fae-cf17-4bdb-baf5-1dde1cd43871/kube-auth-proxy/0.log" Jul 28 18:02:48.279672 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:48.279576 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-vnfxw_ecd3cc49-6ce7-4db3-a0d0-797b1600822b/istio-proxy/0.log" Jul 28 18:02:48.303291 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:48.303268 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-vnfxw_ecd3cc49-6ce7-4db3-a0d0-797b1600822b/istio-proxy/1.log" Jul 28 18:02:48.334356 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:48.334324 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-4n9g5_3c4430fa-54ac-4a06-87fc-08555d52774b/payload-pre-processing/0.log" Jul 28 18:02:48.387297 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:48.387269 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64c477cb5c-9stkr_af095ddd-e1cd-4148-9796-d60be4fef0e2/router/0.log" Jul 28 18:02:48.410025 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:48.410000 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-s9lmn_e988c6ed-5788-4f9c-91d3-263b35ab7d49/manager/0.log" Jul 28 18:02:48.620147 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:48.620063 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-lmn47_039b79ed-2f6b-41ba-8529-fe1a6be83441/manager/0.log" Jul 28 18:02:48.786186 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:48.786159 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7d6f646b66-jww2t_4facf865-9753-4107-82ef-fd66a0bb9e93/manager/0.log" Jul 28 18:02:48.820670 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:48.820645 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-7t86x_9eb4c97f-be9f-4584-b563-9a7e7efc1d2a/server/0.log" Jul 28 18:02:48.919370 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:48.919287 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-mpxpz_650f3a82-4de5-4bd2-ba3b-4f9c4c666f53/manager/0.log" Jul 28 18:02:49.033405 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:49.033371 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d8c7b7dcd-lc72m_4c258043-1b2d-4bd2-abfb-c46a681d788b/manager/0.log" Jul 28 18:02:49.060783 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:49.060758 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d8c7b7dcd-lc72m_4c258043-1b2d-4bd2-abfb-c46a681d788b/copy-manifests/0.log" Jul 28 18:02:49.851492 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:49.851409 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-gbbn7_9bbda9ba-84f0-40a5-979d-9ea282384e4e/istio-proxy/0.log" Jul 28 18:02:50.051572 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:50.051536 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77b678cfbc-gg5gz_1d224fae-cf17-4bdb-baf5-1dde1cd43871/kube-auth-proxy/0.log" Jul 28 18:02:50.083395 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:50.083364 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-vnfxw_ecd3cc49-6ce7-4db3-a0d0-797b1600822b/istio-proxy/0.log" Jul 28 18:02:50.107950 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:50.107882 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-vnfxw_ecd3cc49-6ce7-4db3-a0d0-797b1600822b/istio-proxy/1.log" Jul 28 18:02:50.132011 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:50.131989 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-4n9g5_3c4430fa-54ac-4a06-87fc-08555d52774b/payload-pre-processing/0.log" Jul 28 18:02:50.193600 ip-10-0-132-11 kubenswrapper[2570]: I0728 18:02:50.193572 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64c477cb5c-9stkr_af095ddd-e1cd-4148-9796-d60be4fef0e2/router/0.log"