Jul 14 17:25:55.746491 ip-10-0-132-13 systemd[1]: Starting Kubernetes Kubelet... Jul 14 17:25:56.200034 ip-10-0-132-13 systemd[1]: Started Kubernetes Kubelet. Jul 14 17:25:56.210840 ip-10-0-132-13 kubenswrapper[2554]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 17:25:56.210840 ip-10-0-132-13 kubenswrapper[2554]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 14 17:25:56.210840 ip-10-0-132-13 kubenswrapper[2554]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 17:25:56.210840 ip-10-0-132-13 kubenswrapper[2554]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 14 17:25:56.210840 ip-10-0-132-13 kubenswrapper[2554]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 17:25:56.210840 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.126271 2554 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 14 17:25:56.210840 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.129904 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 17:25:56.210840 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.129916 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 17:25:56.210840 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.129919 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 17:25:56.210840 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.129922 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 17:25:56.210840 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.129926 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 17:25:56.210840 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.129929 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 17:25:56.210840 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.129931 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 17:25:56.210840 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.129934 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 17:25:56.212077 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.129937 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 17:25:56.212077 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.129939 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 17:25:56.212077 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.129942 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 17:25:56.212077 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.129945 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 17:25:56.212077 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.129948 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 17:25:56.212077 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.129950 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 17:25:56.212077 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.129953 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 17:25:56.212077 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.129955 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 17:25:56.212077 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.129965 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 17:25:56.212077 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.129968 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 17:25:56.212077 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.129973 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 17:25:56.212077 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.129977 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 17:25:56.212077 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.129980 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 17:25:56.212077 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.129983 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 17:25:56.212077 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.129986 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 17:25:56.212077 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.129990 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 17:25:56.212077 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.129993 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 17:25:56.212077 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.129996 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 17:25:56.212077 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.129998 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 17:25:56.213105 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130001 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 17:25:56.213105 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130003 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 17:25:56.213105 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130006 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 17:25:56.213105 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130009 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 17:25:56.213105 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130012 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 17:25:56.213105 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130014 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 17:25:56.213105 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130017 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 17:25:56.213105 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130019 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 17:25:56.213105 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130022 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 17:25:56.213105 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130025 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 17:25:56.213105 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130027 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 17:25:56.213105 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130030 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 17:25:56.213105 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130032 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 17:25:56.213105 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130035 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 17:25:56.213105 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130037 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 17:25:56.213105 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130040 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 17:25:56.213105 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130042 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 17:25:56.213105 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130046 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 17:25:56.213105 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130049 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 17:25:56.213105 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130052 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 17:25:56.214726 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130055 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 17:25:56.214726 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130057 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 17:25:56.214726 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130060 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 17:25:56.214726 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130062 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 14 17:25:56.214726 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130065 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 17:25:56.214726 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130067 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 17:25:56.214726 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130070 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 17:25:56.214726 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130073 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 17:25:56.214726 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130076 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 17:25:56.214726 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130078 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 17:25:56.214726 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130081 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 17:25:56.214726 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130083 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 17:25:56.214726 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130086 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 17:25:56.214726 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130088 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 17:25:56.214726 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130091 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 17:25:56.214726 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130093 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 17:25:56.214726 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130096 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 17:25:56.214726 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130099 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 17:25:56.214726 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130101 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 17:25:56.214726 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130106 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 17:25:56.215781 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130109 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 17:25:56.215781 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130112 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 17:25:56.215781 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130115 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 17:25:56.215781 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130118 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 17:25:56.215781 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130121 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 17:25:56.215781 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130124 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 17:25:56.215781 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130127 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 17:25:56.215781 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130129 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 17:25:56.215781 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130132 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 17:25:56.215781 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130135 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 17:25:56.215781 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130138 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 17:25:56.215781 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130141 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 17:25:56.215781 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130144 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 17:25:56.215781 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130147 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 17:25:56.215781 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130150 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 17:25:56.215781 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130153 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 17:25:56.215781 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130156 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 17:25:56.215781 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130158 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 17:25:56.215781 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130161 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 17:25:56.215781 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130163 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 17:25:56.216897 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130166 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 17:25:56.216897 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130168 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 17:25:56.216897 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130171 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 17:25:56.216897 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130174 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 17:25:56.216897 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130176 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 17:25:56.216897 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130178 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 17:25:56.216897 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130181 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 17:25:56.216897 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130184 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 17:25:56.216897 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130188 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 17:25:56.216897 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130192 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 17:25:56.216897 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130195 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 17:25:56.216897 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130198 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 17:25:56.216897 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130888 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 17:25:56.216897 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130895 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 17:25:56.216897 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130897 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 17:25:56.216897 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130900 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 17:25:56.216897 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130904 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 17:25:56.216897 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130907 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 17:25:56.216897 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130909 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 17:25:56.217837 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130912 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 17:25:56.217837 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130915 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 17:25:56.217837 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130917 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 17:25:56.217837 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130920 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 17:25:56.217837 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130923 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 17:25:56.217837 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130925 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 17:25:56.217837 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130928 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 17:25:56.217837 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130932 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 17:25:56.217837 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130935 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 17:25:56.217837 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130938 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 17:25:56.217837 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130940 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 17:25:56.217837 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130943 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 17:25:56.217837 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130946 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 17:25:56.217837 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130948 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 14 17:25:56.217837 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130951 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 17:25:56.217837 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130953 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 17:25:56.217837 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130956 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 17:25:56.217837 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130958 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 17:25:56.217837 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130961 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 17:25:56.217837 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130965 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 17:25:56.219281 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130968 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 17:25:56.219281 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130970 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 17:25:56.219281 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130973 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 17:25:56.219281 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130976 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 17:25:56.219281 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130978 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 17:25:56.219281 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130981 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 17:25:56.219281 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130984 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 17:25:56.219281 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130986 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 17:25:56.219281 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130989 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 17:25:56.219281 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130992 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 17:25:56.219281 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.130996 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 17:25:56.219281 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131000 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 17:25:56.219281 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131002 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 17:25:56.219281 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131005 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 17:25:56.219281 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131008 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 17:25:56.219281 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131011 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 17:25:56.219281 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131013 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 17:25:56.219281 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131017 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 17:25:56.219281 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131019 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 17:25:56.220277 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131022 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 17:25:56.220277 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131025 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 17:25:56.220277 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131028 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 17:25:56.220277 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131031 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 17:25:56.220277 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131034 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 17:25:56.220277 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131036 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 17:25:56.220277 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131039 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 17:25:56.220277 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131041 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 17:25:56.220277 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131045 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 17:25:56.220277 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131049 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 17:25:56.220277 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131053 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 17:25:56.220277 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131056 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 17:25:56.220277 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131058 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 17:25:56.220277 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131061 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 17:25:56.220277 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131063 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 17:25:56.220277 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131065 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 17:25:56.220277 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131068 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 17:25:56.220277 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131070 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 17:25:56.220277 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131073 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 17:25:56.220277 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131075 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 17:25:56.221632 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131078 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 17:25:56.221632 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131080 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 17:25:56.221632 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131083 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 17:25:56.221632 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131085 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 17:25:56.221632 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131087 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 17:25:56.221632 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131090 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 17:25:56.221632 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131092 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 17:25:56.221632 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131097 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 17:25:56.221632 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131099 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 17:25:56.221632 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131102 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 17:25:56.221632 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131105 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 17:25:56.221632 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131107 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 17:25:56.221632 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131110 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 17:25:56.221632 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131113 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 17:25:56.221632 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131116 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 17:25:56.221632 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131118 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 17:25:56.221632 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131122 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 17:25:56.221632 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131125 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 17:25:56.221632 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131127 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 17:25:56.221632 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131130 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 17:25:56.222613 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131132 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 17:25:56.222613 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131135 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 17:25:56.222613 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131138 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 17:25:56.222613 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131140 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 17:25:56.222613 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131143 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 17:25:56.222613 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131145 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 17:25:56.222613 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131147 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 17:25:56.222613 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131150 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 17:25:56.222613 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131152 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 17:25:56.222613 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131155 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 17:25:56.222613 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131157 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 17:25:56.222613 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131161 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 17:25:56.222613 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131165 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 17:25:56.222613 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131548 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 17:25:56.222613 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131554 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 17:25:56.222613 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131558 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 17:25:56.222613 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131561 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 17:25:56.222613 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131565 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 17:25:56.222613 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131569 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 17:25:56.223697 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131575 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 17:25:56.223697 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131578 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 17:25:56.223697 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131582 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 17:25:56.223697 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131585 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 17:25:56.223697 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131587 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 17:25:56.223697 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131590 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 17:25:56.223697 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131593 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 17:25:56.223697 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131596 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 17:25:56.223697 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131598 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 17:25:56.223697 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131601 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 17:25:56.223697 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131603 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 17:25:56.223697 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131606 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 17:25:56.223697 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131608 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 17:25:56.223697 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131611 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 17:25:56.223697 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131613 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 17:25:56.223697 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131616 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 17:25:56.223697 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131618 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 17:25:56.223697 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131621 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 17:25:56.223697 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131624 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 17:25:56.223697 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131626 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 17:25:56.402880 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131629 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 17:25:56.402880 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131632 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 17:25:56.402880 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131634 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 17:25:56.402880 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131637 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 17:25:56.402880 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131639 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 17:25:56.402880 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131642 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 17:25:56.402880 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131644 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 17:25:56.402880 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131647 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 17:25:56.402880 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131649 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 17:25:56.402880 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131652 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 17:25:56.402880 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131654 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 17:25:56.402880 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131657 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 17:25:56.402880 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131661 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 17:25:56.402880 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131664 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 17:25:56.402880 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131673 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 17:25:56.402880 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131675 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 17:25:56.402880 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131678 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 17:25:56.402880 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131680 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 17:25:56.402880 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131683 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 17:25:56.426732 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131685 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 17:25:56.426732 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131694 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 17:25:56.426732 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131696 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 17:25:56.426732 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131699 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 17:25:56.426732 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131701 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 17:25:56.426732 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131704 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 17:25:56.426732 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131706 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 17:25:56.426732 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131709 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 17:25:56.426732 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131712 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 17:25:56.426732 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131714 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 17:25:56.426732 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131717 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 17:25:56.426732 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131719 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 17:25:56.426732 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131722 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 17:25:56.426732 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131724 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 17:25:56.426732 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131726 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 17:25:56.426732 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131729 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 17:25:56.426732 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131731 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 17:25:56.426732 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131734 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 17:25:56.426732 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131736 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 17:25:56.426732 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131739 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 17:25:56.427684 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131741 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 17:25:56.427684 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131743 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 17:25:56.427684 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131746 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 17:25:56.427684 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131748 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 17:25:56.427684 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131751 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 17:25:56.427684 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131755 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 17:25:56.427684 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131759 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 17:25:56.427684 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131763 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 17:25:56.427684 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131767 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 17:25:56.427684 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131769 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 17:25:56.427684 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131772 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 14 17:25:56.427684 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131775 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 17:25:56.427684 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131778 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 17:25:56.427684 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131781 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 17:25:56.427684 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131785 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 17:25:56.427684 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131787 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 17:25:56.427684 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131789 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 17:25:56.427684 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131792 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 17:25:56.427684 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131794 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 17:25:56.427684 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131797 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 17:25:56.428642 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131800 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 17:25:56.428642 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131802 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 17:25:56.428642 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131804 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 17:25:56.428642 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131807 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 17:25:56.428642 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131810 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 17:25:56.428642 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131813 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 17:25:56.428642 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131815 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 17:25:56.428642 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131818 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 17:25:56.428642 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131820 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 17:25:56.428642 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131823 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 17:25:56.428642 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131825 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 17:25:56.428642 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131828 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 17:25:56.428642 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131832 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 17:25:56.428642 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.131835 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 17:25:56.428642 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.131903 2554 flags.go:64] FLAG: --address="0.0.0.0" Jul 14 17:25:56.428642 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.131910 2554 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 14 17:25:56.428642 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.131916 2554 flags.go:64] FLAG: --anonymous-auth="true" Jul 14 17:25:56.428642 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.131923 2554 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 14 17:25:56.428642 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.131929 2554 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 14 17:25:56.428642 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.131933 2554 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 14 17:25:56.428642 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.131939 2554 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 14 17:25:56.429410 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.131945 2554 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 14 17:25:56.429410 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.131948 2554 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 14 17:25:56.429410 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.131951 2554 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 14 17:25:56.429410 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.131954 2554 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 14 17:25:56.429410 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.131958 2554 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 14 17:25:56.429410 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.131960 2554 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 14 17:25:56.429410 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.131964 2554 flags.go:64] FLAG: --cgroup-root="" Jul 14 17:25:56.429410 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.131967 2554 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 14 17:25:56.429410 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.131970 2554 flags.go:64] FLAG: --client-ca-file="" Jul 14 17:25:56.429410 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.131973 2554 flags.go:64] FLAG: --cloud-provider="external" Jul 14 17:25:56.429410 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.131976 2554 flags.go:64] FLAG: --cluster-dns="[]" Jul 14 17:25:56.429410 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.131981 2554 flags.go:64] FLAG: --cluster-domain="" Jul 14 17:25:56.429410 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.131984 2554 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 14 17:25:56.429410 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.131987 2554 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 14 17:25:56.429410 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.131991 2554 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 14 17:25:56.429410 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.131994 2554 flags.go:64] FLAG: --container-log-max-files="5" Jul 14 17:25:56.429410 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.131998 2554 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 14 17:25:56.429410 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132001 2554 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 14 17:25:56.429410 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132004 2554 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 14 17:25:56.429410 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132007 2554 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 14 17:25:56.429410 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132010 2554 flags.go:64] FLAG: --contention-profiling="false" Jul 14 17:25:56.429410 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132013 2554 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 14 17:25:56.429410 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132017 2554 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 14 17:25:56.429410 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132020 2554 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 14 17:25:56.430489 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132023 2554 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 14 17:25:56.430489 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132027 2554 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 14 17:25:56.430489 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132029 2554 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 14 17:25:56.430489 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132033 2554 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 14 17:25:56.430489 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132037 2554 flags.go:64] FLAG: --enable-load-reader="false" Jul 14 17:25:56.430489 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132040 2554 flags.go:64] FLAG: --enable-server="true" Jul 14 17:25:56.430489 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132043 2554 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 14 17:25:56.430489 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132047 2554 flags.go:64] FLAG: --event-burst="100" Jul 14 17:25:56.430489 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132050 2554 flags.go:64] FLAG: --event-qps="50" Jul 14 17:25:56.430489 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132053 2554 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 14 17:25:56.430489 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132056 2554 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 14 17:25:56.430489 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132059 2554 flags.go:64] FLAG: --eviction-hard="" Jul 14 17:25:56.430489 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132063 2554 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 14 17:25:56.430489 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132065 2554 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 14 17:25:56.430489 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132068 2554 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 14 17:25:56.430489 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132072 2554 flags.go:64] FLAG: --eviction-soft="" Jul 14 17:25:56.430489 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132076 2554 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 14 17:25:56.430489 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132079 2554 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 14 17:25:56.430489 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132082 2554 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 14 17:25:56.430489 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132085 2554 flags.go:64] FLAG: --experimental-mounter-path="" Jul 14 17:25:56.430489 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132088 2554 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 14 17:25:56.430489 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132091 2554 flags.go:64] FLAG: --fail-swap-on="true" Jul 14 17:25:56.430489 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132094 2554 flags.go:64] FLAG: --feature-gates="" Jul 14 17:25:56.430489 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132097 2554 flags.go:64] FLAG: --file-check-frequency="20s" Jul 14 17:25:56.430489 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132100 2554 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 14 17:25:56.431352 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132103 2554 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 14 17:25:56.431352 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132106 2554 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 14 17:25:56.431352 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132109 2554 flags.go:64] FLAG: --healthz-port="10248" Jul 14 17:25:56.431352 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132112 2554 flags.go:64] FLAG: --help="false" Jul 14 17:25:56.431352 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132114 2554 flags.go:64] FLAG: --hostname-override="ip-10-0-132-13.ec2.internal" Jul 14 17:25:56.431352 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132117 2554 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 14 17:25:56.431352 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132120 2554 flags.go:64] FLAG: --http-check-frequency="20s" Jul 14 17:25:56.431352 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132123 2554 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 14 17:25:56.431352 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132126 2554 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 14 17:25:56.431352 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132129 2554 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 14 17:25:56.431352 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132132 2554 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 14 17:25:56.431352 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132136 2554 flags.go:64] FLAG: --image-service-endpoint="" Jul 14 17:25:56.431352 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132139 2554 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 14 17:25:56.431352 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132142 2554 flags.go:64] FLAG: --kube-api-burst="100" Jul 14 17:25:56.431352 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132145 2554 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 14 17:25:56.431352 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132148 2554 flags.go:64] FLAG: --kube-api-qps="50" Jul 14 17:25:56.431352 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132151 2554 flags.go:64] FLAG: --kube-reserved="" Jul 14 17:25:56.431352 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132153 2554 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 14 17:25:56.431352 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132156 2554 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 14 17:25:56.431352 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132159 2554 flags.go:64] FLAG: --kubelet-cgroups="" Jul 14 17:25:56.431352 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132162 2554 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 14 17:25:56.431352 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132164 2554 flags.go:64] FLAG: --lock-file="" Jul 14 17:25:56.431352 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132167 2554 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 14 17:25:56.431352 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132172 2554 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 14 17:25:56.432242 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132175 2554 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 14 17:25:56.432242 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132180 2554 flags.go:64] FLAG: --log-json-split-stream="false" Jul 14 17:25:56.432242 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132183 2554 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 14 17:25:56.432242 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132186 2554 flags.go:64] FLAG: --log-text-split-stream="false" Jul 14 17:25:56.432242 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132189 2554 flags.go:64] FLAG: --logging-format="text" Jul 14 17:25:56.432242 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132191 2554 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 14 17:25:56.432242 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132195 2554 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 14 17:25:56.432242 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132198 2554 flags.go:64] FLAG: --manifest-url="" Jul 14 17:25:56.432242 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132205 2554 flags.go:64] FLAG: --manifest-url-header="" Jul 14 17:25:56.432242 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132229 2554 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 14 17:25:56.432242 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132234 2554 flags.go:64] FLAG: --max-open-files="1000000" Jul 14 17:25:56.432242 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132240 2554 flags.go:64] FLAG: --max-pods="110" Jul 14 17:25:56.432242 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132243 2554 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 14 17:25:56.432242 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132246 2554 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 14 17:25:56.432242 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132248 2554 flags.go:64] FLAG: --memory-manager-policy="None" Jul 14 17:25:56.432242 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132251 2554 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 14 17:25:56.432242 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132254 2554 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 14 17:25:56.432242 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132257 2554 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 14 17:25:56.432242 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132260 2554 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 14 17:25:56.432242 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132269 2554 flags.go:64] FLAG: --node-status-max-images="50" Jul 14 17:25:56.432242 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132272 2554 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 14 17:25:56.432242 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132275 2554 flags.go:64] FLAG: --oom-score-adj="-999" Jul 14 17:25:56.432242 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132278 2554 flags.go:64] FLAG: --pod-cidr="" Jul 14 17:25:56.432961 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132281 2554 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 14 17:25:56.432961 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132287 2554 flags.go:64] FLAG: --pod-manifest-path="" Jul 14 17:25:56.432961 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132290 2554 flags.go:64] FLAG: --pod-max-pids="-1" Jul 14 17:25:56.432961 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132293 2554 flags.go:64] FLAG: --pods-per-core="0" Jul 14 17:25:56.432961 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132295 2554 flags.go:64] FLAG: --port="10250" Jul 14 17:25:56.432961 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132298 2554 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 14 17:25:56.432961 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132301 2554 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-038e10cb854170f19" Jul 14 17:25:56.432961 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132304 2554 flags.go:64] FLAG: --qos-reserved="" Jul 14 17:25:56.432961 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132309 2554 flags.go:64] FLAG: --read-only-port="10255" Jul 14 17:25:56.432961 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132312 2554 flags.go:64] FLAG: --register-node="true" Jul 14 17:25:56.432961 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132315 2554 flags.go:64] FLAG: --register-with-taints="" Jul 14 17:25:56.432961 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132319 2554 flags.go:64] FLAG: --registry-burst="10" Jul 14 17:25:56.432961 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132322 2554 flags.go:64] FLAG: --registry-qps="5" Jul 14 17:25:56.432961 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132324 2554 flags.go:64] FLAG: --reserved-cpus="" Jul 14 17:25:56.432961 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132327 2554 flags.go:64] FLAG: --reserved-memory="" Jul 14 17:25:56.432961 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132331 2554 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 14 17:25:56.432961 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132333 2554 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 14 17:25:56.432961 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132337 2554 flags.go:64] FLAG: --rotate-certificates="false" Jul 14 17:25:56.432961 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132340 2554 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 14 17:25:56.432961 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132342 2554 flags.go:64] FLAG: --runonce="false" Jul 14 17:25:56.432961 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132345 2554 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 14 17:25:56.432961 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132348 2554 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 14 17:25:56.432961 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132351 2554 flags.go:64] FLAG: --seccomp-default="false" Jul 14 17:25:56.432961 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132354 2554 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 14 17:25:56.432961 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132357 2554 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 14 17:25:56.433875 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132360 2554 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 14 17:25:56.433875 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132362 2554 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 14 17:25:56.433875 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132365 2554 flags.go:64] FLAG: --storage-driver-password="root" Jul 14 17:25:56.433875 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132368 2554 flags.go:64] FLAG: --storage-driver-secure="false" Jul 14 17:25:56.433875 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132372 2554 flags.go:64] FLAG: --storage-driver-table="stats" Jul 14 17:25:56.433875 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132375 2554 flags.go:64] FLAG: --storage-driver-user="root" Jul 14 17:25:56.433875 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132377 2554 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 14 17:25:56.433875 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132381 2554 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 14 17:25:56.433875 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132383 2554 flags.go:64] FLAG: --system-cgroups="" Jul 14 17:25:56.433875 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132386 2554 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 14 17:25:56.433875 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132391 2554 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 14 17:25:56.433875 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132394 2554 flags.go:64] FLAG: --tls-cert-file="" Jul 14 17:25:56.433875 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132396 2554 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 14 17:25:56.433875 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132401 2554 flags.go:64] FLAG: --tls-min-version="" Jul 14 17:25:56.433875 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132403 2554 flags.go:64] FLAG: --tls-private-key-file="" Jul 14 17:25:56.433875 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132408 2554 flags.go:64] FLAG: --topology-manager-policy="none" Jul 14 17:25:56.433875 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132411 2554 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 14 17:25:56.433875 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132415 2554 flags.go:64] FLAG: --topology-manager-scope="container" Jul 14 17:25:56.433875 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132417 2554 flags.go:64] FLAG: --v="2" Jul 14 17:25:56.433875 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132421 2554 flags.go:64] FLAG: --version="false" Jul 14 17:25:56.433875 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132425 2554 flags.go:64] FLAG: --vmodule="" Jul 14 17:25:56.433875 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132429 2554 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 14 17:25:56.433875 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.132432 2554 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 14 17:25:56.433875 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132555 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 17:25:56.433875 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132560 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 17:25:56.434641 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132563 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 17:25:56.434641 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132566 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 17:25:56.434641 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132570 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 17:25:56.434641 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132572 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 17:25:56.434641 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132575 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 17:25:56.434641 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132578 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 17:25:56.434641 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132581 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 17:25:56.434641 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132583 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 17:25:56.434641 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132586 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 17:25:56.434641 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132588 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 17:25:56.434641 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132591 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 17:25:56.434641 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132595 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 17:25:56.434641 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132597 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 17:25:56.434641 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132600 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 17:25:56.434641 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132602 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 17:25:56.434641 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132605 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 17:25:56.434641 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132608 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 17:25:56.434641 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132610 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 17:25:56.434641 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132613 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 17:25:56.434641 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132615 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 17:25:56.435515 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132618 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 17:25:56.435515 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132621 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 14 17:25:56.435515 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132625 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 17:25:56.435515 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132627 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 17:25:56.435515 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132630 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 17:25:56.435515 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132634 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 17:25:56.435515 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132636 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 17:25:56.435515 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132639 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 17:25:56.435515 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132641 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 17:25:56.435515 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132644 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 17:25:56.435515 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132646 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 17:25:56.435515 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132649 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 17:25:56.435515 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132651 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 17:25:56.435515 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132653 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 17:25:56.435515 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132656 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 17:25:56.435515 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132658 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 17:25:56.435515 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132661 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 17:25:56.435515 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132664 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 17:25:56.435515 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132666 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 17:25:56.435515 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132668 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 17:25:56.436125 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132671 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 17:25:56.436125 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132674 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 17:25:56.436125 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132676 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 17:25:56.436125 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132680 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 17:25:56.436125 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132682 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 17:25:56.436125 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132685 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 17:25:56.436125 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132687 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 17:25:56.436125 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132690 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 17:25:56.436125 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132692 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 17:25:56.436125 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132695 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 17:25:56.436125 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132697 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 17:25:56.436125 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132700 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 17:25:56.436125 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132702 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 17:25:56.436125 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132705 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 17:25:56.436125 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132708 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 17:25:56.436125 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132711 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 17:25:56.436125 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132713 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 17:25:56.436125 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132716 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 17:25:56.436125 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132719 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 17:25:56.436125 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132721 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 17:25:56.740155 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132724 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 17:25:56.740155 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132726 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 17:25:56.740155 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132729 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 17:25:56.740155 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132731 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 17:25:56.740155 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132734 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 17:25:56.740155 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132736 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 17:25:56.740155 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132739 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 17:25:56.740155 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132741 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 17:25:56.740155 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132744 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 17:25:56.740155 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132746 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 17:25:56.740155 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132748 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 17:25:56.740155 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132751 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 17:25:56.740155 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132753 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 17:25:56.740155 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132756 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 17:25:56.740155 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132760 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 17:25:56.740155 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132764 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 17:25:56.740155 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132767 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 17:25:56.740155 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132770 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 17:25:56.740155 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132781 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 17:25:56.741320 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132784 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 17:25:56.741320 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132788 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 17:25:56.741320 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132791 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 17:25:56.741320 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132794 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 17:25:56.741320 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132797 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 17:25:56.741320 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132800 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 17:25:56.741320 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132803 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 17:25:56.741320 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132808 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 17:25:56.741320 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132810 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 17:25:56.741320 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132813 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 17:25:56.741320 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132816 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 17:25:56.741320 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132818 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 17:25:56.741320 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132822 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 17:25:56.741320 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132824 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 17:25:56.741320 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132827 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 17:25:56.741320 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132829 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 17:25:56.741320 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132833 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 17:25:56.741320 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.132836 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 17:25:56.741963 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.133361 2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 17:25:56.741963 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.143629 2554 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 14 17:25:56.741963 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.143643 2554 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 14 17:25:56.741963 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.143663 2554 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 14 17:25:56.741963 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.143672 2554 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 14 17:25:56.741963 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143730 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 17:25:56.741963 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143736 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 17:25:56.741963 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143739 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 17:25:56.741963 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143742 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 17:25:56.741963 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143746 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 17:25:56.741963 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143749 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 17:25:56.741963 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143752 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 17:25:56.741963 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143755 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 17:25:56.741963 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143758 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 17:25:56.741963 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143760 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 17:25:56.742749 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143763 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 17:25:56.742749 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143765 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 17:25:56.742749 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143768 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 17:25:56.742749 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143771 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 17:25:56.742749 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143773 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 17:25:56.742749 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143776 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 17:25:56.742749 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143778 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 17:25:56.742749 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143781 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 17:25:56.742749 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143784 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 17:25:56.742749 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143786 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 17:25:56.742749 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143789 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 17:25:56.742749 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143792 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 17:25:56.742749 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143796 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 17:25:56.742749 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143800 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 17:25:56.742749 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143803 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 17:25:56.742749 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143808 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 17:25:56.742749 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143812 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 17:25:56.742749 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143815 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 17:25:56.742749 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143818 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 17:25:56.743339 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143822 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 17:25:56.743339 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143825 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 17:25:56.743339 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143827 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 17:25:56.743339 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143830 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 17:25:56.743339 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143832 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 17:25:56.743339 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143835 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 17:25:56.743339 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143837 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 17:25:56.743339 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143840 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 17:25:56.743339 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143843 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 17:25:56.743339 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143846 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 17:25:56.743339 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143848 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 17:25:56.743339 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143851 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 17:25:56.743339 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143854 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 17:25:56.743339 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143856 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 17:25:56.743339 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143859 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 17:25:56.743339 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143862 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 17:25:56.743339 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143864 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 17:25:56.743339 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143867 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 17:25:56.743339 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143870 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 17:25:56.743983 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143873 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 17:25:56.743983 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143876 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 17:25:56.743983 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143878 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 17:25:56.743983 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143881 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 17:25:56.743983 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143884 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 17:25:56.743983 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143886 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 17:25:56.743983 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143889 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 17:25:56.743983 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143892 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 17:25:56.743983 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143894 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 17:25:56.743983 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143896 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 17:25:56.743983 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143899 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 17:25:56.743983 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143902 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 17:25:56.743983 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143904 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 17:25:56.743983 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143907 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 17:25:56.743983 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143910 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 17:25:56.743983 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143912 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 17:25:56.743983 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143916 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 17:25:56.743983 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143920 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 17:25:56.743983 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143922 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 17:25:56.743983 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143925 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 17:25:56.744738 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143928 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 17:25:56.744738 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143930 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 17:25:56.744738 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143933 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 17:25:56.744738 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143935 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 17:25:56.744738 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143938 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 17:25:56.744738 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143940 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 17:25:56.744738 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143943 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 17:25:56.744738 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143946 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 17:25:56.744738 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143948 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 17:25:56.744738 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143951 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 17:25:56.744738 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143954 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 17:25:56.744738 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143956 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 17:25:56.744738 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143961 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 17:25:56.744738 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143964 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 17:25:56.744738 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143967 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 17:25:56.744738 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143969 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 17:25:56.744738 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143972 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 17:25:56.744738 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143975 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 17:25:56.744738 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143977 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 17:25:56.744738 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143980 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 14 17:25:56.745374 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143982 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 17:25:56.745374 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143986 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 17:25:56.745374 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143989 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 17:25:56.745374 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143992 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 17:25:56.745374 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143994 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 17:25:56.745374 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143997 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 17:25:56.745374 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.143999 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 17:25:56.745374 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144003 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 17:25:56.745374 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144005 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 17:25:56.745374 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144008 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 17:25:56.745374 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144011 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 17:25:56.745374 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.144016 2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 17:25:56.745374 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144672 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 17:25:56.745374 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144678 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 17:25:56.745374 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144682 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 17:25:56.746037 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144685 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 17:25:56.746037 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144687 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 17:25:56.746037 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144690 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 17:25:56.746037 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144693 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 17:25:56.746037 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144695 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 17:25:56.746037 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144698 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 17:25:56.746037 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144701 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 17:25:56.746037 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144703 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 17:25:56.746037 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144706 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 17:25:56.746037 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144708 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 17:25:56.746037 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144714 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 17:25:56.746037 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144717 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 17:25:56.746037 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144719 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 17:25:56.746037 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144722 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 17:25:56.746037 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144724 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 17:25:56.746037 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144727 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 17:25:56.746037 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144730 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 17:25:56.746037 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144732 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 17:25:56.746037 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144736 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 17:25:56.746037 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144739 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 17:25:56.746628 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144741 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 14 17:25:56.746628 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144744 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 17:25:56.746628 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144746 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 17:25:56.746628 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144749 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 17:25:56.746628 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144751 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 17:25:56.746628 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144755 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 17:25:56.746628 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144757 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 17:25:56.746628 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144760 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 17:25:56.746628 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144762 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 17:25:56.746628 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144764 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 17:25:56.746628 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144767 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 17:25:56.746628 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144769 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 17:25:56.746628 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144780 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 17:25:56.746628 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144783 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 17:25:56.746628 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144786 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 17:25:56.746628 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144788 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 17:25:56.746628 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144791 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 17:25:56.746628 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144794 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 17:25:56.746628 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144796 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 17:25:56.746628 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144799 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 17:25:56.908581 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144801 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 17:25:56.908581 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144804 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 17:25:56.908581 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144808 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 17:25:56.908581 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144812 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 17:25:56.908581 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144816 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 17:25:56.908581 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144819 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 17:25:56.908581 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144822 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 17:25:56.908581 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144825 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 17:25:56.908581 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144827 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 17:25:56.908581 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144830 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 17:25:56.908581 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144833 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 17:25:56.908581 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144836 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 17:25:56.908581 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144838 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 17:25:56.908581 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144841 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 17:25:56.908581 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144843 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 17:25:56.908581 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144846 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 17:25:56.908581 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144848 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 17:25:56.908581 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144851 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 17:25:56.908581 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144856 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 17:25:56.909931 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144859 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 17:25:56.909931 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144862 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 17:25:56.909931 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144865 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 17:25:56.909931 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144867 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 17:25:56.909931 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144870 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 17:25:56.909931 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144873 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 17:25:56.909931 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144875 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 17:25:56.909931 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144878 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 17:25:56.909931 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144880 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 17:25:56.909931 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144883 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 17:25:56.909931 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144886 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 17:25:56.909931 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144888 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 17:25:56.909931 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144890 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 17:25:56.909931 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144893 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 17:25:56.909931 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144896 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 17:25:56.909931 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144899 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 17:25:56.909931 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144902 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 17:25:56.909931 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144904 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 17:25:56.909931 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144908 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 17:25:57.075872 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144911 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 17:25:57.075872 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144914 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 17:25:57.075872 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144917 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 17:25:57.075872 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144920 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 17:25:57.075872 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144924 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 17:25:57.075872 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144927 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 17:25:57.075872 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144930 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 17:25:57.075872 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144932 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 17:25:57.075872 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144935 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 17:25:57.075872 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144937 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 17:25:57.075872 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144940 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 17:25:57.075872 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144942 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 17:25:57.075872 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144945 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 17:25:57.075872 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144948 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 17:25:57.075872 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144950 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 17:25:57.075872 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144953 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 17:25:57.075872 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144955 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 17:25:57.075872 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:56.144958 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 17:25:57.076984 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.144962 2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 17:25:57.076984 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.145058 2554 server.go:962] "Client rotation is on, will bootstrap in background" Jul 14 17:25:57.076984 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.147042 2554 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 14 17:25:57.076984 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.148324 2554 server.go:1019] "Starting client certificate rotation" Jul 14 17:25:57.076984 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.148413 2554 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 17:25:57.076984 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.148440 2554 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 17:25:57.076984 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.150649 2554 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 17:25:57.076984 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.156626 2554 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 17:25:57.076984 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.167975 2554 log.go:25] "Validated CRI v1 runtime API" Jul 14 17:25:57.076984 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.170961 2554 log.go:25] "Validated CRI v1 image API" Jul 14 17:25:57.076984 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.172429 2554 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 14 17:25:57.076984 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.175942 2554 fs.go:135] Filesystem UUIDs: map[2c58b6ef-e4f3-46ab-ab11-a2fcac960bd6:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 95375328-c92e-4a25-8226-fabc1cf12efc:/dev/nvme0n1p3] Jul 14 17:25:57.077769 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.175959 2554 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 14 17:25:57.078946 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.181795 2554 manager.go:216] Machine: {Timestamp:2026-07-14 17:25:56.181086437 +0000 UTC m=+0.322335136 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099748 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2918a78a81967d43392a000b759175 SystemUUID:ec2918a7-8a81-967d-4339-2a000b759175 BootID:007ebdd9-0419-4934-8f30-1491c6022435 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:3d:74:6f:1c:a5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:3d:74:6f:1c:a5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:06:2e:45:42:86:61 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 14 17:25:57.078946 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.181935 2554 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 14 17:25:57.078946 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.182066 2554 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 14 17:25:57.078946 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.183225 2554 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 17:25:57.078946 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.184627 2554 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 14 17:25:57.078946 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.184652 2554 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-13.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 14 17:25:57.078946 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.185313 2554 topology_manager.go:138] "Creating topology manager with none policy" Jul 14 17:25:57.079347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.185323 2554 container_manager_linux.go:309] "Creating device plugin manager" Jul 14 17:25:57.079347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.185331 2554 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 17:25:57.079347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.185343 2554 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 17:25:57.079347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.187243 2554 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 14 17:25:57.079347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.190008 2554 state_mem.go:36] "Initialized new in-memory state store" Jul 14 17:25:57.079347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.190125 2554 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 14 17:25:57.079347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.190154 2554 kubelet.go:486] "Attempting to sync node with API server" Jul 14 17:25:57.079347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.190164 2554 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 14 17:25:57.079347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.190177 2554 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 14 17:25:57.079347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.190186 2554 kubelet.go:398] "Adding apiserver pod source" Jul 14 17:25:57.079347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.190194 2554 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 14 17:25:57.079347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.190536 2554 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 17:25:57.079347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.190551 2554 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 17:25:57.079347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.192489 2554 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 14 17:25:57.079347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.194645 2554 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 14 17:25:57.079347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.194672 2554 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 14 17:25:57.079347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.194844 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 14 17:25:57.079347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.194861 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 14 17:25:57.079347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.194867 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 14 17:25:57.079347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.194875 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 14 17:25:57.079347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.194886 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 14 17:25:57.079347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.194892 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 14 17:25:57.079347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.195265 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 14 17:25:57.079347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.195271 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 14 17:25:57.079347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.195278 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 14 17:25:57.079347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.195283 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 14 17:25:57.079347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.195291 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 14 17:25:57.079347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.195298 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 14 17:25:57.079347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.195327 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 14 17:25:57.079347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.195333 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 14 17:25:57.079347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.198436 2554 server.go:1268] "Started kubelet" Jul 14 17:25:57.079347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.198525 2554 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 14 17:25:57.079347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.198710 2554 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 14 17:25:57.079347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.198866 2554 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 14 17:25:57.079347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.200062 2554 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 14 17:25:57.079347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.201307 2554 server.go:310] "Adding debug handlers to kubelet server" Jul 14 17:25:57.079347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.203637 2554 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-13.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 14 17:25:57.079347 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:56.203639 2554 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-13.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 14 17:25:57.079347 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:56.203680 2554 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 14 17:25:57.081283 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:56.204441 2554 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 14 17:25:57.081283 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.205669 2554 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 14 17:25:57.081283 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:56.205749 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-13.ec2.internal\" not found" Jul 14 17:25:57.081283 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.205923 2554 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 14 17:25:57.081283 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.206252 2554 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 14 17:25:57.081283 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.206270 2554 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 14 17:25:57.081283 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.206346 2554 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 14 17:25:57.081283 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.206549 2554 reconstruct.go:97] "Volume reconstruction finished" Jul 14 17:25:57.081283 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.206560 2554 reconciler.go:29] "Reconciler: start to sync state" Jul 14 17:25:57.081283 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.206701 2554 factory.go:55] Registering systemd factory Jul 14 17:25:57.081283 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.206723 2554 factory.go:223] Registration of the systemd container factory successfully Jul 14 17:25:57.081283 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.206990 2554 factory.go:153] Registering CRI-O factory Jul 14 17:25:57.081283 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.207007 2554 factory.go:223] Registration of the crio container factory successfully Jul 14 17:25:57.081283 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.207079 2554 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 14 17:25:57.081283 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.207122 2554 factory.go:103] Registering Raw factory Jul 14 17:25:57.081283 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.207149 2554 manager.go:1196] Started watching for new ooms in manager Jul 14 17:25:57.081283 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:56.203936 2554 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-13.ec2.internal.18c237fb9cc1ec51 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-13.ec2.internal,UID:ip-10-0-132-13.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-13.ec2.internal,},FirstTimestamp:2026-07-14 17:25:56.198411345 +0000 UTC m=+0.339660039,LastTimestamp:2026-07-14 17:25:56.198411345 +0000 UTC m=+0.339660039,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-13.ec2.internal,}" Jul 14 17:25:57.081283 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.208163 2554 manager.go:349] Starting recovery of all containers Jul 14 17:25:57.081283 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.208588 2554 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-b9cfl" Jul 14 17:25:57.081283 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:56.212179 2554 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-13.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 14 17:25:57.081283 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:56.212280 2554 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 14 17:25:57.081283 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.216542 2554 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-b9cfl" Jul 14 17:25:57.081283 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.221676 2554 manager.go:354] Recovery completed Jul 14 17:25:57.081283 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.226126 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 17:25:57.081283 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.227788 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-13.ec2.internal" event="NodeHasSufficientMemory" Jul 14 17:25:57.081283 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.227814 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-13.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 17:25:57.081283 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.227826 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-13.ec2.internal" event="NodeHasSufficientPID" Jul 14 17:25:57.081283 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.228268 2554 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 14 17:25:57.081283 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.228278 2554 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 14 17:25:57.081283 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.228277 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 17:25:57.081283 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.228320 2554 state_mem.go:36] "Initialized new in-memory state store" Jul 14 17:25:57.081283 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.228992 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-13.ec2.internal" event="NodeHasSufficientMemory" Jul 14 17:25:57.082473 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.229014 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-13.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 17:25:57.082473 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.229025 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-13.ec2.internal" event="NodeHasSufficientPID" Jul 14 17:25:57.082473 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.229302 2554 policy_none.go:49] "None policy: Start" Jul 14 17:25:57.082473 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.229314 2554 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 14 17:25:57.082473 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.229323 2554 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 14 17:25:57.082473 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:56.229620 2554 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-13.ec2.internal.18c237fb9e82564c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-13.ec2.internal,UID:ip-10-0-132-13.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-13.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-13.ec2.internal,},FirstTimestamp:2026-07-14 17:25:56.227798604 +0000 UTC m=+0.369047299,LastTimestamp:2026-07-14 17:25:56.227798604 +0000 UTC m=+0.369047299,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-13.ec2.internal,}" Jul 14 17:25:57.082473 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.230373 2554 policy_none.go:47] "Start" Jul 14 17:25:57.082473 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.264714 2554 manager.go:337] "Starting Device Plugin manager" Jul 14 17:25:57.082473 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:56.264847 2554 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 14 17:25:57.082473 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.264857 2554 server.go:87] "Starting device plugin registration server" Jul 14 17:25:57.082473 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.265047 2554 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 14 17:25:57.082473 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.265056 2554 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 14 17:25:57.082473 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.265164 2554 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 14 17:25:57.082473 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.265256 2554 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 14 17:25:57.082473 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.265262 2554 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 14 17:25:57.082473 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:56.265705 2554 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 14 17:25:57.082473 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:56.265757 2554 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-13.ec2.internal\" not found" Jul 14 17:25:57.082473 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.365659 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 17:25:57.082473 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.366480 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-13.ec2.internal" event="NodeHasSufficientMemory" Jul 14 17:25:57.082473 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.366506 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-13.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 17:25:57.082473 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.366515 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-13.ec2.internal" event="NodeHasSufficientPID" Jul 14 17:25:57.082473 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.366539 2554 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-13.ec2.internal" Jul 14 17:25:57.082473 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.374819 2554 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-13.ec2.internal" Jul 14 17:25:57.082473 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:56.374838 2554 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-13.ec2.internal\": node \"ip-10-0-132-13.ec2.internal\" not found" Jul 14 17:25:57.082473 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.379476 2554 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 14 17:25:57.082473 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.381343 2554 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 14 17:25:57.082473 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.381383 2554 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 14 17:25:57.082473 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.381408 2554 kubelet.go:2448] "Starting kubelet main sync loop" Jul 14 17:25:57.082473 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:56.381450 2554 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 14 17:25:57.082473 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.383600 2554 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 17:25:57.082473 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:56.388025 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-13.ec2.internal\" not found" Jul 14 17:25:57.082473 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.485307 2554 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-13.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-13.ec2.internal"] Jul 14 17:25:57.082473 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.485374 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 17:25:57.084030 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.486182 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-13.ec2.internal" event="NodeHasSufficientMemory" Jul 14 17:25:57.084030 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.486206 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-13.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 17:25:57.084030 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.486230 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-13.ec2.internal" event="NodeHasSufficientPID" Jul 14 17:25:57.084030 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.487453 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 17:25:57.084030 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.487605 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-13.ec2.internal" Jul 14 17:25:57.084030 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.487632 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 17:25:57.084030 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:56.488079 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-13.ec2.internal\" not found" Jul 14 17:25:57.084030 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.488087 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-13.ec2.internal" event="NodeHasSufficientMemory" Jul 14 17:25:57.084030 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.488117 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-13.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 17:25:57.084030 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.488125 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-13.ec2.internal" event="NodeHasSufficientMemory" Jul 14 17:25:57.084030 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.488146 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-13.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 17:25:57.084030 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.488156 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-13.ec2.internal" event="NodeHasSufficientPID" Jul 14 17:25:57.084030 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.488127 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-13.ec2.internal" event="NodeHasSufficientPID" Jul 14 17:25:57.084030 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.489401 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-13.ec2.internal" Jul 14 17:25:57.084030 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.489424 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 17:25:57.084030 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.490098 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-13.ec2.internal" event="NodeHasSufficientMemory" Jul 14 17:25:57.084030 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.490129 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-13.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 17:25:57.084030 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.490142 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-13.ec2.internal" event="NodeHasSufficientPID" Jul 14 17:25:57.084030 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.509701 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b3ea3cd1054689f0ec1aa0dca0048bb1-config\") pod \"kube-apiserver-proxy-ip-10-0-132-13.ec2.internal\" (UID: \"b3ea3cd1054689f0ec1aa0dca0048bb1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-13.ec2.internal" Jul 14 17:25:57.084030 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.509721 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/93bf474703caedf9ae57eedb84b07448-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-13.ec2.internal\" (UID: \"93bf474703caedf9ae57eedb84b07448\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-13.ec2.internal" Jul 14 17:25:57.084030 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.509741 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/93bf474703caedf9ae57eedb84b07448-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-13.ec2.internal\" (UID: \"93bf474703caedf9ae57eedb84b07448\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-13.ec2.internal" Jul 14 17:25:57.084030 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:56.510856 2554 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-13.ec2.internal\" not found" node="ip-10-0-132-13.ec2.internal" Jul 14 17:25:57.084030 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:56.512165 2554 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-13.ec2.internal\" not found" node="ip-10-0-132-13.ec2.internal" Jul 14 17:25:57.084030 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:56.588139 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-13.ec2.internal\" not found" Jul 14 17:25:57.084030 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.610024 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b3ea3cd1054689f0ec1aa0dca0048bb1-config\") pod \"kube-apiserver-proxy-ip-10-0-132-13.ec2.internal\" (UID: \"b3ea3cd1054689f0ec1aa0dca0048bb1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-13.ec2.internal" Jul 14 17:25:57.086452 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.610063 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/93bf474703caedf9ae57eedb84b07448-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-13.ec2.internal\" (UID: \"93bf474703caedf9ae57eedb84b07448\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-13.ec2.internal" Jul 14 17:25:57.086452 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.610080 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/93bf474703caedf9ae57eedb84b07448-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-13.ec2.internal\" (UID: \"93bf474703caedf9ae57eedb84b07448\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-13.ec2.internal" Jul 14 17:25:57.086452 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.610107 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/93bf474703caedf9ae57eedb84b07448-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-13.ec2.internal\" (UID: \"93bf474703caedf9ae57eedb84b07448\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-13.ec2.internal" Jul 14 17:25:57.086452 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.610127 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/93bf474703caedf9ae57eedb84b07448-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-13.ec2.internal\" (UID: \"93bf474703caedf9ae57eedb84b07448\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-13.ec2.internal" Jul 14 17:25:57.086452 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.610125 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b3ea3cd1054689f0ec1aa0dca0048bb1-config\") pod \"kube-apiserver-proxy-ip-10-0-132-13.ec2.internal\" (UID: \"b3ea3cd1054689f0ec1aa0dca0048bb1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-13.ec2.internal" Jul 14 17:25:57.086452 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:56.690737 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-13.ec2.internal\" not found" Jul 14 17:25:57.086452 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:56.797135 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-13.ec2.internal\" not found" Jul 14 17:25:57.086452 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.814876 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-13.ec2.internal" Jul 14 17:25:57.086452 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:56.816041 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-13.ec2.internal" Jul 14 17:25:57.086452 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:56.898934 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-13.ec2.internal\" not found" Jul 14 17:25:57.086452 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:57.006377 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-13.ec2.internal\" not found" Jul 14 17:25:57.109956 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:57.109919 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-13.ec2.internal\" not found" Jul 14 17:25:57.113551 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.113527 2554 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 17:25:57.149136 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.149116 2554 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 14 17:25:57.149301 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.149274 2554 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 14 17:25:57.149342 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.149313 2554 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 14 17:25:57.191188 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.191160 2554 apiserver.go:52] "Watching apiserver" Jul 14 17:25:57.206256 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.206236 2554 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-13.ec2.internal" Jul 14 17:25:57.206342 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.206256 2554 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 14 17:25:57.209573 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.209555 2554 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 14 17:25:57.210690 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.210671 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-b8gnk","openshift-image-registry/node-ca-7ffq4","openshift-multus/multus-additional-cni-plugins-k9kwq","openshift-multus/network-metrics-daemon-ctxxr","openshift-ovn-kubernetes/ovnkube-node-pk9h2","kube-system/global-pull-secret-syncer-88sg9","kube-system/konnectivity-agent-djms6","openshift-cluster-node-tuning-operator/tuned-pqnvp","openshift-multus/multus-ct2sx","openshift-network-diagnostics/network-check-target-4lhqc","openshift-network-operator/iptables-alerter-c7d6p","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf9sf"] Jul 14 17:25:57.212368 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.212350 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-b8gnk" Jul 14 17:25:57.213791 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.213775 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7ffq4" Jul 14 17:25:57.214998 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.214980 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-k9kwq" Jul 14 17:25:57.215086 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.215010 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 14 17:25:57.215277 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.215189 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-bv78g\"" Jul 14 17:25:57.215277 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.215238 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 14 17:25:57.216153 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.216130 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctxxr" Jul 14 17:25:57.216245 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:57.216201 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctxxr" podUID="b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb" Jul 14 17:25:57.216320 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.216275 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 14 17:25:57.216463 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.216443 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 14 17:25:57.216524 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.216451 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-j4hfm\"" Jul 14 17:25:57.216524 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.216495 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 14 17:25:57.217503 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.217486 2554 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 17:25:57.217596 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.217556 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 14 17:25:57.217596 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.217589 2554 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-13.ec2.internal" Jul 14 17:25:57.217870 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.217849 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fknmv\"" Jul 14 17:25:57.217955 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.217946 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 14 17:25:57.218017 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.217963 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 14 17:25:57.218405 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.218381 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.218494 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.218471 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 14 17:25:57.218494 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.218484 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 14 17:25:57.220616 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.220137 2554 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 17:25:57.220616 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.220145 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:25:57.221001 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:57.220970 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88sg9" podUID="69371a04-a55d-4548-a7df-2868cd880309" Jul 14 17:25:57.221374 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.221354 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 14 17:25:57.221456 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.221437 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 14 17:25:57.221506 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.221473 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-lll87\"" Jul 14 17:25:57.221953 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.221932 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 14 17:25:57.222213 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.222197 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 14 17:25:57.222419 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.222404 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 14 17:25:57.222627 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.222611 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 14 17:25:57.222712 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.222651 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-djms6" Jul 14 17:25:57.224403 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.224385 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.224624 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.224564 2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-13 17:20:56 +0000 UTC" deadline="2028-02-05 02:57:57.270227718 +0000 UTC" Jul 14 17:25:57.224624 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.224620 2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13689h32m0.045619109s" Jul 14 17:25:57.224795 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.224779 2554 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 17:25:57.225410 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.225392 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-vdhb7\"" Jul 14 17:25:57.225482 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.225424 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 14 17:25:57.225482 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.225429 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 14 17:25:57.225661 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.225646 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.226717 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.226696 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 14 17:25:57.226822 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.226804 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-fs5tj\"" Jul 14 17:25:57.226872 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.226817 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 14 17:25:57.227047 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.227033 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4lhqc" Jul 14 17:25:57.227129 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:57.227110 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4lhqc" podUID="56fc04ea-db5c-4d02-a7ff-a92cf718cb15" Jul 14 17:25:57.227826 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.227810 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-l6d42\"" Jul 14 17:25:57.227980 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.227963 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 14 17:25:57.228493 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.228479 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-c7d6p" Jul 14 17:25:57.230032 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.230012 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-13.ec2.internal"] Jul 14 17:25:57.230117 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.230039 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-132-13.ec2.internal"] Jul 14 17:25:57.230117 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.230097 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf9sf" Jul 14 17:25:57.230890 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.230875 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 14 17:25:57.231327 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.231308 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4js4s\"" Jul 14 17:25:57.231425 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.231389 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 14 17:25:57.231425 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.231401 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 14 17:25:57.232303 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.232282 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 14 17:25:57.232604 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.232571 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 14 17:25:57.232706 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.232631 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-lzxkm\"" Jul 14 17:25:57.232787 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.232776 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 14 17:25:57.240077 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.240061 2554 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-gmnvv" Jul 14 17:25:57.247586 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.247564 2554 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-gmnvv" Jul 14 17:25:57.307443 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.307425 2554 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 14 17:25:57.348060 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348038 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/814adb62-03ce-494d-9e1f-1b78f45b87e2-os-release\") pod \"multus-additional-cni-plugins-k9kwq\" (UID: \"814adb62-03ce-494d-9e1f-1b78f45b87e2\") " pod="openshift-multus/multus-additional-cni-plugins-k9kwq" Jul 14 17:25:57.348157 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348064 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/69371a04-a55d-4548-a7df-2868cd880309-kubelet-config\") pod \"global-pull-secret-syncer-88sg9\" (UID: \"69371a04-a55d-4548-a7df-2868cd880309\") " pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:25:57.348157 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348083 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/69392349-f201-43dd-9748-78613041a9a9-etc-kubernetes\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.348157 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348098 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/69392349-f201-43dd-9748-78613041a9a9-etc-sysctl-conf\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.348157 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348149 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-os-release\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.348301 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348176 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-hostroot\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.348301 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348192 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-multus-daemon-config\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.348301 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348216 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25t4c\" (UniqueName: \"kubernetes.io/projected/814adb62-03ce-494d-9e1f-1b78f45b87e2-kube-api-access-25t4c\") pod \"multus-additional-cni-plugins-k9kwq\" (UID: \"814adb62-03ce-494d-9e1f-1b78f45b87e2\") " pod="openshift-multus/multus-additional-cni-plugins-k9kwq" Jul 14 17:25:57.348301 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348250 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-host-run-netns\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.348301 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348265 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-ovn-node-metrics-cert\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.348301 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348280 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-ovnkube-script-lib\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.348507 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348305 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/69392349-f201-43dd-9748-78613041a9a9-etc-modprobe-d\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.348507 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348334 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/69392349-f201-43dd-9748-78613041a9a9-etc-tuned\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.348507 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348353 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwrkj\" (UniqueName: \"kubernetes.io/projected/7f7817bc-7352-4f6a-9f20-cf101d130a7c-kube-api-access-xwrkj\") pod \"node-ca-7ffq4\" (UID: \"7f7817bc-7352-4f6a-9f20-cf101d130a7c\") " pod="openshift-image-registry/node-ca-7ffq4" Jul 14 17:25:57.348507 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348372 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsxs8\" (UniqueName: \"kubernetes.io/projected/b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb-kube-api-access-qsxs8\") pod \"network-metrics-daemon-ctxxr\" (UID: \"b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb\") " pod="openshift-multus/network-metrics-daemon-ctxxr" Jul 14 17:25:57.348507 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348392 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nq99q\" (UniqueName: \"kubernetes.io/projected/56fc04ea-db5c-4d02-a7ff-a92cf718cb15-kube-api-access-nq99q\") pod \"network-check-target-4lhqc\" (UID: \"56fc04ea-db5c-4d02-a7ff-a92cf718cb15\") " pod="openshift-network-diagnostics/network-check-target-4lhqc" Jul 14 17:25:57.348507 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348414 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dc814136-ef4b-4cb5-a9c1-dd078e83b2f1-sys-fs\") pod \"aws-ebs-csi-driver-node-hf9sf\" (UID: \"dc814136-ef4b-4cb5-a9c1-dd078e83b2f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf9sf" Jul 14 17:25:57.348507 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348431 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzqfd\" (UniqueName: \"kubernetes.io/projected/69392349-f201-43dd-9748-78613041a9a9-kube-api-access-bzqfd\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.348507 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348447 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-multus-cni-dir\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.348507 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348461 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-cni-binary-copy\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.348507 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348499 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2k62s\" (UniqueName: \"kubernetes.io/projected/7768649b-c82e-4a04-a4c1-93061292bcec-kube-api-access-2k62s\") pod \"node-resolver-b8gnk\" (UID: \"7768649b-c82e-4a04-a4c1-93061292bcec\") " pod="openshift-dns/node-resolver-b8gnk" Jul 14 17:25:57.348840 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348514 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dc814136-ef4b-4cb5-a9c1-dd078e83b2f1-device-dir\") pod \"aws-ebs-csi-driver-node-hf9sf\" (UID: \"dc814136-ef4b-4cb5-a9c1-dd078e83b2f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf9sf" Jul 14 17:25:57.348840 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348528 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-var-lib-openvswitch\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.348840 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348553 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.348840 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348577 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-ovnkube-config\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.348840 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348596 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/69371a04-a55d-4548-a7df-2868cd880309-dbus\") pod \"global-pull-secret-syncer-88sg9\" (UID: \"69371a04-a55d-4548-a7df-2868cd880309\") " pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:25:57.348840 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348610 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/69392349-f201-43dd-9748-78613041a9a9-lib-modules\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.348840 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348628 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-host-var-lib-cni-bin\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.348840 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348652 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-host-run-k8s-cni-cncf-io\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.348840 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348671 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-etc-kubernetes\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.348840 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348685 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gktpm\" (UniqueName: \"kubernetes.io/projected/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-kube-api-access-gktpm\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.348840 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348706 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dc814136-ef4b-4cb5-a9c1-dd078e83b2f1-etc-selinux\") pod \"aws-ebs-csi-driver-node-hf9sf\" (UID: \"dc814136-ef4b-4cb5-a9c1-dd078e83b2f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf9sf" Jul 14 17:25:57.348840 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348723 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-host-kubelet\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.348840 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348740 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/69392349-f201-43dd-9748-78613041a9a9-var-lib-kubelet\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.348840 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348753 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-run-ovn\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.349486 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348768 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-log-socket\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.349486 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348787 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-host-cni-netd\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.349486 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348811 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/69392349-f201-43dd-9748-78613041a9a9-host\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.349486 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348832 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-cnibin\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.349486 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348854 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgf79\" (UniqueName: \"kubernetes.io/projected/dc814136-ef4b-4cb5-a9c1-dd078e83b2f1-kube-api-access-tgf79\") pod \"aws-ebs-csi-driver-node-hf9sf\" (UID: \"dc814136-ef4b-4cb5-a9c1-dd078e83b2f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf9sf" Jul 14 17:25:57.349486 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348872 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-host-run-ovn-kubernetes\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.349486 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348897 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/69392349-f201-43dd-9748-78613041a9a9-etc-sysctl-d\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.349486 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348931 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/210c935e-f5d4-4de0-abee-7d5e2f555f34-konnectivity-ca\") pod \"konnectivity-agent-djms6\" (UID: \"210c935e-f5d4-4de0-abee-7d5e2f555f34\") " pod="kube-system/konnectivity-agent-djms6" Jul 14 17:25:57.349486 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348949 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/814adb62-03ce-494d-9e1f-1b78f45b87e2-system-cni-dir\") pod \"multus-additional-cni-plugins-k9kwq\" (UID: \"814adb62-03ce-494d-9e1f-1b78f45b87e2\") " pod="openshift-multus/multus-additional-cni-plugins-k9kwq" Jul 14 17:25:57.349486 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348965 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/814adb62-03ce-494d-9e1f-1b78f45b87e2-cni-binary-copy\") pod \"multus-additional-cni-plugins-k9kwq\" (UID: \"814adb62-03ce-494d-9e1f-1b78f45b87e2\") " pod="openshift-multus/multus-additional-cni-plugins-k9kwq" Jul 14 17:25:57.349486 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.348982 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/814adb62-03ce-494d-9e1f-1b78f45b87e2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k9kwq\" (UID: \"814adb62-03ce-494d-9e1f-1b78f45b87e2\") " pod="openshift-multus/multus-additional-cni-plugins-k9kwq" Jul 14 17:25:57.349486 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.349007 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-run-openvswitch\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.349486 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.349030 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-node-log\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.349947 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.349047 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69371a04-a55d-4548-a7df-2868cd880309-original-pull-secret\") pod \"global-pull-secret-syncer-88sg9\" (UID: \"69371a04-a55d-4548-a7df-2868cd880309\") " pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:25:57.349947 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.349061 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-multus-socket-dir-parent\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.349947 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.349076 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/210c935e-f5d4-4de0-abee-7d5e2f555f34-agent-certs\") pod \"konnectivity-agent-djms6\" (UID: \"210c935e-f5d4-4de0-abee-7d5e2f555f34\") " pod="kube-system/konnectivity-agent-djms6" Jul 14 17:25:57.349947 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.349096 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7768649b-c82e-4a04-a4c1-93061292bcec-tmp-dir\") pod \"node-resolver-b8gnk\" (UID: \"7768649b-c82e-4a04-a4c1-93061292bcec\") " pod="openshift-dns/node-resolver-b8gnk" Jul 14 17:25:57.349947 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.349125 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7f7817bc-7352-4f6a-9f20-cf101d130a7c-host\") pod \"node-ca-7ffq4\" (UID: \"7f7817bc-7352-4f6a-9f20-cf101d130a7c\") " pod="openshift-image-registry/node-ca-7ffq4" Jul 14 17:25:57.349947 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.349144 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7f7817bc-7352-4f6a-9f20-cf101d130a7c-serviceca\") pod \"node-ca-7ffq4\" (UID: \"7f7817bc-7352-4f6a-9f20-cf101d130a7c\") " pod="openshift-image-registry/node-ca-7ffq4" Jul 14 17:25:57.349947 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.349158 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-run-systemd\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.349947 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.349191 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/69392349-f201-43dd-9748-78613041a9a9-run\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.349947 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.349232 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/69392349-f201-43dd-9748-78613041a9a9-sys\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.349947 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.349255 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-host-var-lib-kubelet\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.349947 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.349274 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-system-cni-dir\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.349947 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.349292 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-host-run-netns\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.349947 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.349307 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d2bc074a-026c-402d-ac70-7d39ff8dc81f-iptables-alerter-script\") pod \"iptables-alerter-c7d6p\" (UID: \"d2bc074a-026c-402d-ac70-7d39ff8dc81f\") " pod="openshift-network-operator/iptables-alerter-c7d6p" Jul 14 17:25:57.349947 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.349324 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52svk\" (UniqueName: \"kubernetes.io/projected/d2bc074a-026c-402d-ac70-7d39ff8dc81f-kube-api-access-52svk\") pod \"iptables-alerter-c7d6p\" (UID: \"d2bc074a-026c-402d-ac70-7d39ff8dc81f\") " pod="openshift-network-operator/iptables-alerter-c7d6p" Jul 14 17:25:57.350401 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.349340 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/814adb62-03ce-494d-9e1f-1b78f45b87e2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k9kwq\" (UID: \"814adb62-03ce-494d-9e1f-1b78f45b87e2\") " pod="openshift-multus/multus-additional-cni-plugins-k9kwq" Jul 14 17:25:57.350401 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.349356 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb-metrics-certs\") pod \"network-metrics-daemon-ctxxr\" (UID: \"b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb\") " pod="openshift-multus/network-metrics-daemon-ctxxr" Jul 14 17:25:57.350401 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.349373 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7768649b-c82e-4a04-a4c1-93061292bcec-hosts-file\") pod \"node-resolver-b8gnk\" (UID: \"7768649b-c82e-4a04-a4c1-93061292bcec\") " pod="openshift-dns/node-resolver-b8gnk" Jul 14 17:25:57.350401 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.349394 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dc814136-ef4b-4cb5-a9c1-dd078e83b2f1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hf9sf\" (UID: \"dc814136-ef4b-4cb5-a9c1-dd078e83b2f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf9sf" Jul 14 17:25:57.350401 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.349418 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dc814136-ef4b-4cb5-a9c1-dd078e83b2f1-socket-dir\") pod \"aws-ebs-csi-driver-node-hf9sf\" (UID: \"dc814136-ef4b-4cb5-a9c1-dd078e83b2f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf9sf" Jul 14 17:25:57.350401 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.349444 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-etc-openvswitch\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.350401 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.349467 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-env-overrides\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.350401 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.349495 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlg74\" (UniqueName: \"kubernetes.io/projected/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-kube-api-access-mlg74\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.350401 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.349528 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-systemd-units\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.350401 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.349555 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-host-cni-bin\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.350401 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.349574 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/69392349-f201-43dd-9748-78613041a9a9-etc-sysconfig\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.350401 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.349597 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-host-var-lib-cni-multus\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.350401 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.349637 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/814adb62-03ce-494d-9e1f-1b78f45b87e2-cnibin\") pod \"multus-additional-cni-plugins-k9kwq\" (UID: \"814adb62-03ce-494d-9e1f-1b78f45b87e2\") " pod="openshift-multus/multus-additional-cni-plugins-k9kwq" Jul 14 17:25:57.350837 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.349679 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/814adb62-03ce-494d-9e1f-1b78f45b87e2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k9kwq\" (UID: \"814adb62-03ce-494d-9e1f-1b78f45b87e2\") " pod="openshift-multus/multus-additional-cni-plugins-k9kwq" Jul 14 17:25:57.350837 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.349702 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-host-slash\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.350837 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.349726 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/69392349-f201-43dd-9748-78613041a9a9-tmp\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.350837 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.349743 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d2bc074a-026c-402d-ac70-7d39ff8dc81f-host-slash\") pod \"iptables-alerter-c7d6p\" (UID: \"d2bc074a-026c-402d-ac70-7d39ff8dc81f\") " pod="openshift-network-operator/iptables-alerter-c7d6p" Jul 14 17:25:57.350837 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.349760 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dc814136-ef4b-4cb5-a9c1-dd078e83b2f1-registration-dir\") pod \"aws-ebs-csi-driver-node-hf9sf\" (UID: \"dc814136-ef4b-4cb5-a9c1-dd078e83b2f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf9sf" Jul 14 17:25:57.350837 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.349774 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/69392349-f201-43dd-9748-78613041a9a9-etc-systemd\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.350837 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.349788 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-multus-conf-dir\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.350837 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.349802 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-host-run-multus-certs\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.450327 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.450292 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/69392349-f201-43dd-9748-78613041a9a9-etc-sysctl-d\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.450327 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.450330 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/210c935e-f5d4-4de0-abee-7d5e2f555f34-konnectivity-ca\") pod \"konnectivity-agent-djms6\" (UID: \"210c935e-f5d4-4de0-abee-7d5e2f555f34\") " pod="kube-system/konnectivity-agent-djms6" Jul 14 17:25:57.450581 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.450347 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/814adb62-03ce-494d-9e1f-1b78f45b87e2-system-cni-dir\") pod \"multus-additional-cni-plugins-k9kwq\" (UID: \"814adb62-03ce-494d-9e1f-1b78f45b87e2\") " pod="openshift-multus/multus-additional-cni-plugins-k9kwq" Jul 14 17:25:57.450581 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.450364 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/814adb62-03ce-494d-9e1f-1b78f45b87e2-cni-binary-copy\") pod \"multus-additional-cni-plugins-k9kwq\" (UID: \"814adb62-03ce-494d-9e1f-1b78f45b87e2\") " pod="openshift-multus/multus-additional-cni-plugins-k9kwq" Jul 14 17:25:57.450581 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.450384 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/814adb62-03ce-494d-9e1f-1b78f45b87e2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k9kwq\" (UID: \"814adb62-03ce-494d-9e1f-1b78f45b87e2\") " pod="openshift-multus/multus-additional-cni-plugins-k9kwq" Jul 14 17:25:57.450581 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.450441 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/814adb62-03ce-494d-9e1f-1b78f45b87e2-system-cni-dir\") pod \"multus-additional-cni-plugins-k9kwq\" (UID: \"814adb62-03ce-494d-9e1f-1b78f45b87e2\") " pod="openshift-multus/multus-additional-cni-plugins-k9kwq" Jul 14 17:25:57.450581 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.450468 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/69392349-f201-43dd-9748-78613041a9a9-etc-sysctl-d\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.450581 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.450507 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-run-openvswitch\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.450581 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.450545 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-node-log\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.450581 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.450577 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69371a04-a55d-4548-a7df-2868cd880309-original-pull-secret\") pod \"global-pull-secret-syncer-88sg9\" (UID: \"69371a04-a55d-4548-a7df-2868cd880309\") " pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:25:57.451078 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.450593 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-run-openvswitch\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.451078 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.450609 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-multus-socket-dir-parent\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.451078 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.450652 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-node-log\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.451078 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.450683 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/210c935e-f5d4-4de0-abee-7d5e2f555f34-agent-certs\") pod \"konnectivity-agent-djms6\" (UID: \"210c935e-f5d4-4de0-abee-7d5e2f555f34\") " pod="kube-system/konnectivity-agent-djms6" Jul 14 17:25:57.451078 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.450710 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7768649b-c82e-4a04-a4c1-93061292bcec-tmp-dir\") pod \"node-resolver-b8gnk\" (UID: \"7768649b-c82e-4a04-a4c1-93061292bcec\") " pod="openshift-dns/node-resolver-b8gnk" Jul 14 17:25:57.451078 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:57.450727 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 17:25:57.451078 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.450734 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7f7817bc-7352-4f6a-9f20-cf101d130a7c-host\") pod \"node-ca-7ffq4\" (UID: \"7f7817bc-7352-4f6a-9f20-cf101d130a7c\") " pod="openshift-image-registry/node-ca-7ffq4" Jul 14 17:25:57.451078 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.450735 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-multus-socket-dir-parent\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.451078 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.450788 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7f7817bc-7352-4f6a-9f20-cf101d130a7c-host\") pod \"node-ca-7ffq4\" (UID: \"7f7817bc-7352-4f6a-9f20-cf101d130a7c\") " pod="openshift-image-registry/node-ca-7ffq4" Jul 14 17:25:57.451078 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:57.450852 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69371a04-a55d-4548-a7df-2868cd880309-original-pull-secret podName:69371a04-a55d-4548-a7df-2868cd880309 nodeName:}" failed. No retries permitted until 2026-07-14 17:25:57.950790045 +0000 UTC m=+2.092038750 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/69371a04-a55d-4548-a7df-2868cd880309-original-pull-secret") pod "global-pull-secret-syncer-88sg9" (UID: "69371a04-a55d-4548-a7df-2868cd880309") : object "kube-system"/"original-pull-secret" not registered Jul 14 17:25:57.451078 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.450877 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7f7817bc-7352-4f6a-9f20-cf101d130a7c-serviceca\") pod \"node-ca-7ffq4\" (UID: \"7f7817bc-7352-4f6a-9f20-cf101d130a7c\") " pod="openshift-image-registry/node-ca-7ffq4" Jul 14 17:25:57.451078 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.450908 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-run-systemd\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.451078 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.450935 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/69392349-f201-43dd-9748-78613041a9a9-run\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.451078 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.450961 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/69392349-f201-43dd-9748-78613041a9a9-sys\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.451078 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.450988 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-host-var-lib-kubelet\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.451908 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451022 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7768649b-c82e-4a04-a4c1-93061292bcec-tmp-dir\") pod \"node-resolver-b8gnk\" (UID: \"7768649b-c82e-4a04-a4c1-93061292bcec\") " pod="openshift-dns/node-resolver-b8gnk" Jul 14 17:25:57.451908 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451032 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-system-cni-dir\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.451908 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451054 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/814adb62-03ce-494d-9e1f-1b78f45b87e2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k9kwq\" (UID: \"814adb62-03ce-494d-9e1f-1b78f45b87e2\") " pod="openshift-multus/multus-additional-cni-plugins-k9kwq" Jul 14 17:25:57.451908 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451061 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-host-run-netns\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.451908 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451068 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/210c935e-f5d4-4de0-abee-7d5e2f555f34-konnectivity-ca\") pod \"konnectivity-agent-djms6\" (UID: \"210c935e-f5d4-4de0-abee-7d5e2f555f34\") " pod="kube-system/konnectivity-agent-djms6" Jul 14 17:25:57.451908 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451094 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d2bc074a-026c-402d-ac70-7d39ff8dc81f-iptables-alerter-script\") pod \"iptables-alerter-c7d6p\" (UID: \"d2bc074a-026c-402d-ac70-7d39ff8dc81f\") " pod="openshift-network-operator/iptables-alerter-c7d6p" Jul 14 17:25:57.451908 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451099 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/814adb62-03ce-494d-9e1f-1b78f45b87e2-cni-binary-copy\") pod \"multus-additional-cni-plugins-k9kwq\" (UID: \"814adb62-03ce-494d-9e1f-1b78f45b87e2\") " pod="openshift-multus/multus-additional-cni-plugins-k9kwq" Jul 14 17:25:57.451908 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451118 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-run-systemd\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.451908 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451122 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52svk\" (UniqueName: \"kubernetes.io/projected/d2bc074a-026c-402d-ac70-7d39ff8dc81f-kube-api-access-52svk\") pod \"iptables-alerter-c7d6p\" (UID: \"d2bc074a-026c-402d-ac70-7d39ff8dc81f\") " pod="openshift-network-operator/iptables-alerter-c7d6p" Jul 14 17:25:57.451908 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451162 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/69392349-f201-43dd-9748-78613041a9a9-run\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.451908 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451175 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-host-run-netns\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.451908 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451177 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/69392349-f201-43dd-9748-78613041a9a9-sys\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.451908 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451186 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-host-var-lib-kubelet\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.451908 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451094 2554 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 14 17:25:57.451908 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451175 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/814adb62-03ce-494d-9e1f-1b78f45b87e2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k9kwq\" (UID: \"814adb62-03ce-494d-9e1f-1b78f45b87e2\") " pod="openshift-multus/multus-additional-cni-plugins-k9kwq" Jul 14 17:25:57.452767 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451234 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb-metrics-certs\") pod \"network-metrics-daemon-ctxxr\" (UID: \"b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb\") " pod="openshift-multus/network-metrics-daemon-ctxxr" Jul 14 17:25:57.452767 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451236 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-system-cni-dir\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.452767 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451260 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7768649b-c82e-4a04-a4c1-93061292bcec-hosts-file\") pod \"node-resolver-b8gnk\" (UID: \"7768649b-c82e-4a04-a4c1-93061292bcec\") " pod="openshift-dns/node-resolver-b8gnk" Jul 14 17:25:57.452767 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451297 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dc814136-ef4b-4cb5-a9c1-dd078e83b2f1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hf9sf\" (UID: \"dc814136-ef4b-4cb5-a9c1-dd078e83b2f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf9sf" Jul 14 17:25:57.452767 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451325 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dc814136-ef4b-4cb5-a9c1-dd078e83b2f1-socket-dir\") pod \"aws-ebs-csi-driver-node-hf9sf\" (UID: \"dc814136-ef4b-4cb5-a9c1-dd078e83b2f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf9sf" Jul 14 17:25:57.452767 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451336 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7768649b-c82e-4a04-a4c1-93061292bcec-hosts-file\") pod \"node-resolver-b8gnk\" (UID: \"7768649b-c82e-4a04-a4c1-93061292bcec\") " pod="openshift-dns/node-resolver-b8gnk" Jul 14 17:25:57.452767 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451350 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-etc-openvswitch\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.452767 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451375 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/814adb62-03ce-494d-9e1f-1b78f45b87e2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k9kwq\" (UID: \"814adb62-03ce-494d-9e1f-1b78f45b87e2\") " pod="openshift-multus/multus-additional-cni-plugins-k9kwq" Jul 14 17:25:57.452767 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451377 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-env-overrides\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.452767 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:57.451422 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:25:57.452767 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:57.451487 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb-metrics-certs podName:b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb nodeName:}" failed. No retries permitted until 2026-07-14 17:25:57.951468581 +0000 UTC m=+2.092717267 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb-metrics-certs") pod "network-metrics-daemon-ctxxr" (UID: "b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:25:57.452767 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451420 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlg74\" (UniqueName: \"kubernetes.io/projected/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-kube-api-access-mlg74\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.452767 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451526 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-systemd-units\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.452767 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451553 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-host-cni-bin\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.453460 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451579 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7f7817bc-7352-4f6a-9f20-cf101d130a7c-serviceca\") pod \"node-ca-7ffq4\" (UID: \"7f7817bc-7352-4f6a-9f20-cf101d130a7c\") " pod="openshift-image-registry/node-ca-7ffq4" Jul 14 17:25:57.453460 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451593 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/69392349-f201-43dd-9748-78613041a9a9-etc-sysconfig\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.453460 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451632 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-host-var-lib-cni-multus\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.453460 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451639 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-host-cni-bin\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.453460 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451647 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-systemd-units\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.453460 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451655 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/814adb62-03ce-494d-9e1f-1b78f45b87e2-cnibin\") pod \"multus-additional-cni-plugins-k9kwq\" (UID: \"814adb62-03ce-494d-9e1f-1b78f45b87e2\") " pod="openshift-multus/multus-additional-cni-plugins-k9kwq" Jul 14 17:25:57.453460 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451690 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/69392349-f201-43dd-9748-78613041a9a9-etc-sysconfig\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.453460 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451708 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-etc-openvswitch\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.453460 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451729 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dc814136-ef4b-4cb5-a9c1-dd078e83b2f1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hf9sf\" (UID: \"dc814136-ef4b-4cb5-a9c1-dd078e83b2f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf9sf" Jul 14 17:25:57.453460 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451741 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/814adb62-03ce-494d-9e1f-1b78f45b87e2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k9kwq\" (UID: \"814adb62-03ce-494d-9e1f-1b78f45b87e2\") " pod="openshift-multus/multus-additional-cni-plugins-k9kwq" Jul 14 17:25:57.453460 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451767 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/814adb62-03ce-494d-9e1f-1b78f45b87e2-cnibin\") pod \"multus-additional-cni-plugins-k9kwq\" (UID: \"814adb62-03ce-494d-9e1f-1b78f45b87e2\") " pod="openshift-multus/multus-additional-cni-plugins-k9kwq" Jul 14 17:25:57.453460 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451773 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-host-slash\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.453460 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451777 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dc814136-ef4b-4cb5-a9c1-dd078e83b2f1-socket-dir\") pod \"aws-ebs-csi-driver-node-hf9sf\" (UID: \"dc814136-ef4b-4cb5-a9c1-dd078e83b2f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf9sf" Jul 14 17:25:57.453460 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451745 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d2bc074a-026c-402d-ac70-7d39ff8dc81f-iptables-alerter-script\") pod \"iptables-alerter-c7d6p\" (UID: \"d2bc074a-026c-402d-ac70-7d39ff8dc81f\") " pod="openshift-network-operator/iptables-alerter-c7d6p" Jul 14 17:25:57.454328 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451733 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-host-var-lib-cni-multus\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.454328 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451798 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/69392349-f201-43dd-9748-78613041a9a9-tmp\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.454328 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451823 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d2bc074a-026c-402d-ac70-7d39ff8dc81f-host-slash\") pod \"iptables-alerter-c7d6p\" (UID: \"d2bc074a-026c-402d-ac70-7d39ff8dc81f\") " pod="openshift-network-operator/iptables-alerter-c7d6p" Jul 14 17:25:57.454328 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451832 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-host-slash\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.454328 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451845 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-env-overrides\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.454328 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451847 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dc814136-ef4b-4cb5-a9c1-dd078e83b2f1-registration-dir\") pod \"aws-ebs-csi-driver-node-hf9sf\" (UID: \"dc814136-ef4b-4cb5-a9c1-dd078e83b2f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf9sf" Jul 14 17:25:57.454328 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451872 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d2bc074a-026c-402d-ac70-7d39ff8dc81f-host-slash\") pod \"iptables-alerter-c7d6p\" (UID: \"d2bc074a-026c-402d-ac70-7d39ff8dc81f\") " pod="openshift-network-operator/iptables-alerter-c7d6p" Jul 14 17:25:57.454328 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451891 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dc814136-ef4b-4cb5-a9c1-dd078e83b2f1-registration-dir\") pod \"aws-ebs-csi-driver-node-hf9sf\" (UID: \"dc814136-ef4b-4cb5-a9c1-dd078e83b2f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf9sf" Jul 14 17:25:57.454328 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451899 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/69392349-f201-43dd-9748-78613041a9a9-etc-systemd\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.454328 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451927 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-multus-conf-dir\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.454328 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451952 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-host-run-multus-certs\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.454328 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451990 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-multus-conf-dir\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.454328 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.451995 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/69392349-f201-43dd-9748-78613041a9a9-etc-systemd\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.454328 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452007 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/814adb62-03ce-494d-9e1f-1b78f45b87e2-os-release\") pod \"multus-additional-cni-plugins-k9kwq\" (UID: \"814adb62-03ce-494d-9e1f-1b78f45b87e2\") " pod="openshift-multus/multus-additional-cni-plugins-k9kwq" Jul 14 17:25:57.454328 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452034 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-host-run-multus-certs\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.455207 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452034 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/69371a04-a55d-4548-a7df-2868cd880309-kubelet-config\") pod \"global-pull-secret-syncer-88sg9\" (UID: \"69371a04-a55d-4548-a7df-2868cd880309\") " pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:25:57.455207 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452065 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/69392349-f201-43dd-9748-78613041a9a9-etc-kubernetes\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.455207 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452084 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/69392349-f201-43dd-9748-78613041a9a9-etc-sysctl-conf\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.455207 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452087 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/69371a04-a55d-4548-a7df-2868cd880309-kubelet-config\") pod \"global-pull-secret-syncer-88sg9\" (UID: \"69371a04-a55d-4548-a7df-2868cd880309\") " pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:25:57.455207 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452112 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-os-release\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.455207 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452109 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/814adb62-03ce-494d-9e1f-1b78f45b87e2-os-release\") pod \"multus-additional-cni-plugins-k9kwq\" (UID: \"814adb62-03ce-494d-9e1f-1b78f45b87e2\") " pod="openshift-multus/multus-additional-cni-plugins-k9kwq" Jul 14 17:25:57.455207 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452125 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/69392349-f201-43dd-9748-78613041a9a9-etc-kubernetes\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.455207 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452139 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-hostroot\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.455207 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452174 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-hostroot\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.455207 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452178 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-multus-daemon-config\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.455207 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452200 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/814adb62-03ce-494d-9e1f-1b78f45b87e2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k9kwq\" (UID: \"814adb62-03ce-494d-9e1f-1b78f45b87e2\") " pod="openshift-multus/multus-additional-cni-plugins-k9kwq" Jul 14 17:25:57.455207 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452208 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25t4c\" (UniqueName: \"kubernetes.io/projected/814adb62-03ce-494d-9e1f-1b78f45b87e2-kube-api-access-25t4c\") pod \"multus-additional-cni-plugins-k9kwq\" (UID: \"814adb62-03ce-494d-9e1f-1b78f45b87e2\") " pod="openshift-multus/multus-additional-cni-plugins-k9kwq" Jul 14 17:25:57.455207 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452267 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-host-run-netns\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.455207 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452294 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-ovn-node-metrics-cert\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.455207 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452180 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-os-release\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.456022 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452321 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-ovnkube-script-lib\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.456022 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452385 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-host-run-netns\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.456022 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452207 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/69392349-f201-43dd-9748-78613041a9a9-etc-sysctl-conf\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.456022 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452457 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/69392349-f201-43dd-9748-78613041a9a9-etc-modprobe-d\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.456022 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452481 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/69392349-f201-43dd-9748-78613041a9a9-etc-tuned\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.456022 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452504 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwrkj\" (UniqueName: \"kubernetes.io/projected/7f7817bc-7352-4f6a-9f20-cf101d130a7c-kube-api-access-xwrkj\") pod \"node-ca-7ffq4\" (UID: \"7f7817bc-7352-4f6a-9f20-cf101d130a7c\") " pod="openshift-image-registry/node-ca-7ffq4" Jul 14 17:25:57.456022 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452531 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsxs8\" (UniqueName: \"kubernetes.io/projected/b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb-kube-api-access-qsxs8\") pod \"network-metrics-daemon-ctxxr\" (UID: \"b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb\") " pod="openshift-multus/network-metrics-daemon-ctxxr" Jul 14 17:25:57.456022 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452557 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq99q\" (UniqueName: \"kubernetes.io/projected/56fc04ea-db5c-4d02-a7ff-a92cf718cb15-kube-api-access-nq99q\") pod \"network-check-target-4lhqc\" (UID: \"56fc04ea-db5c-4d02-a7ff-a92cf718cb15\") " pod="openshift-network-diagnostics/network-check-target-4lhqc" Jul 14 17:25:57.456022 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452585 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dc814136-ef4b-4cb5-a9c1-dd078e83b2f1-sys-fs\") pod \"aws-ebs-csi-driver-node-hf9sf\" (UID: \"dc814136-ef4b-4cb5-a9c1-dd078e83b2f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf9sf" Jul 14 17:25:57.456022 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452615 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzqfd\" (UniqueName: \"kubernetes.io/projected/69392349-f201-43dd-9748-78613041a9a9-kube-api-access-bzqfd\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.456022 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452644 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-multus-cni-dir\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.456022 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452669 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-cni-binary-copy\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.456022 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452693 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2k62s\" (UniqueName: \"kubernetes.io/projected/7768649b-c82e-4a04-a4c1-93061292bcec-kube-api-access-2k62s\") pod \"node-resolver-b8gnk\" (UID: \"7768649b-c82e-4a04-a4c1-93061292bcec\") " pod="openshift-dns/node-resolver-b8gnk" Jul 14 17:25:57.456022 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452723 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dc814136-ef4b-4cb5-a9c1-dd078e83b2f1-device-dir\") pod \"aws-ebs-csi-driver-node-hf9sf\" (UID: \"dc814136-ef4b-4cb5-a9c1-dd078e83b2f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf9sf" Jul 14 17:25:57.456495 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452728 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-multus-daemon-config\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.456495 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452614 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/69392349-f201-43dd-9748-78613041a9a9-etc-modprobe-d\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.456495 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452767 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-var-lib-openvswitch\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.456495 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452793 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.456495 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452818 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-ovnkube-config\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.456495 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452844 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/69371a04-a55d-4548-a7df-2868cd880309-dbus\") pod \"global-pull-secret-syncer-88sg9\" (UID: \"69371a04-a55d-4548-a7df-2868cd880309\") " pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:25:57.456495 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452870 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/69392349-f201-43dd-9748-78613041a9a9-lib-modules\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.456495 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452896 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-host-var-lib-cni-bin\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.456495 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452921 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-ovnkube-script-lib\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.456495 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452930 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dc814136-ef4b-4cb5-a9c1-dd078e83b2f1-sys-fs\") pod \"aws-ebs-csi-driver-node-hf9sf\" (UID: \"dc814136-ef4b-4cb5-a9c1-dd078e83b2f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf9sf" Jul 14 17:25:57.456495 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452924 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-host-run-k8s-cni-cncf-io\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.456495 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.452992 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-etc-kubernetes\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.456495 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.453027 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gktpm\" (UniqueName: \"kubernetes.io/projected/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-kube-api-access-gktpm\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.456495 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.453056 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dc814136-ef4b-4cb5-a9c1-dd078e83b2f1-etc-selinux\") pod \"aws-ebs-csi-driver-node-hf9sf\" (UID: \"dc814136-ef4b-4cb5-a9c1-dd078e83b2f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf9sf" Jul 14 17:25:57.456910 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.453084 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-etc-kubernetes\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.456910 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.453092 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-host-kubelet\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.456910 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.453137 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/69392349-f201-43dd-9748-78613041a9a9-var-lib-kubelet\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.456910 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.453142 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-host-kubelet\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.456910 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.453165 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-run-ovn\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.456910 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.453190 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-log-socket\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.456910 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.453234 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dc814136-ef4b-4cb5-a9c1-dd078e83b2f1-device-dir\") pod \"aws-ebs-csi-driver-node-hf9sf\" (UID: \"dc814136-ef4b-4cb5-a9c1-dd078e83b2f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf9sf" Jul 14 17:25:57.456910 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.453241 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-host-cni-netd\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.456910 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.453301 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-host-cni-netd\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.456910 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.453305 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/69392349-f201-43dd-9748-78613041a9a9-host\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.456910 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.453341 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-cnibin\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.456910 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.453351 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/69392349-f201-43dd-9748-78613041a9a9-host\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.456910 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.453371 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgf79\" (UniqueName: \"kubernetes.io/projected/dc814136-ef4b-4cb5-a9c1-dd078e83b2f1-kube-api-access-tgf79\") pod \"aws-ebs-csi-driver-node-hf9sf\" (UID: \"dc814136-ef4b-4cb5-a9c1-dd078e83b2f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf9sf" Jul 14 17:25:57.456910 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.453397 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-var-lib-openvswitch\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.456910 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.453402 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-host-run-ovn-kubernetes\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.457388 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.453444 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-host-run-ovn-kubernetes\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.457388 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.453467 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dc814136-ef4b-4cb5-a9c1-dd078e83b2f1-etc-selinux\") pod \"aws-ebs-csi-driver-node-hf9sf\" (UID: \"dc814136-ef4b-4cb5-a9c1-dd078e83b2f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf9sf" Jul 14 17:25:57.457388 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.453492 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-host-var-lib-cni-bin\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.457388 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.453518 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-log-socket\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.457388 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.453549 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/69392349-f201-43dd-9748-78613041a9a9-var-lib-kubelet\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.457388 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.453561 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-run-ovn\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.457388 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.453560 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-multus-cni-dir\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.457388 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.453559 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/69371a04-a55d-4548-a7df-2868cd880309-dbus\") pod \"global-pull-secret-syncer-88sg9\" (UID: \"69371a04-a55d-4548-a7df-2868cd880309\") " pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:25:57.457388 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.453606 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.457388 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.453659 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-host-run-k8s-cni-cncf-io\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.457388 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.453704 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-cnibin\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.457388 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.453834 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/69392349-f201-43dd-9748-78613041a9a9-lib-modules\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.457388 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.454110 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-ovnkube-config\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.457388 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.454531 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-cni-binary-copy\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.457388 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.455048 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/69392349-f201-43dd-9748-78613041a9a9-tmp\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.457829 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.455181 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/69392349-f201-43dd-9748-78613041a9a9-etc-tuned\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.457829 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.455371 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/210c935e-f5d4-4de0-abee-7d5e2f555f34-agent-certs\") pod \"konnectivity-agent-djms6\" (UID: \"210c935e-f5d4-4de0-abee-7d5e2f555f34\") " pod="kube-system/konnectivity-agent-djms6" Jul 14 17:25:57.457829 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.456263 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-ovn-node-metrics-cert\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.462077 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:57.461890 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 17:25:57.462077 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:57.461920 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 17:25:57.462077 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:57.461934 2554 projected.go:196] Error preparing data for projected volume kube-api-access-nq99q for pod openshift-network-diagnostics/network-check-target-4lhqc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:25:57.462077 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:57.462018 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/56fc04ea-db5c-4d02-a7ff-a92cf718cb15-kube-api-access-nq99q podName:56fc04ea-db5c-4d02-a7ff-a92cf718cb15 nodeName:}" failed. No retries permitted until 2026-07-14 17:25:57.961994031 +0000 UTC m=+2.103242736 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-nq99q" (UniqueName: "kubernetes.io/projected/56fc04ea-db5c-4d02-a7ff-a92cf718cb15-kube-api-access-nq99q") pod "network-check-target-4lhqc" (UID: "56fc04ea-db5c-4d02-a7ff-a92cf718cb15") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:25:57.462908 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.462686 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlg74\" (UniqueName: \"kubernetes.io/projected/f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726-kube-api-access-mlg74\") pod \"ovnkube-node-pk9h2\" (UID: \"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.464405 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.464376 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgf79\" (UniqueName: \"kubernetes.io/projected/dc814136-ef4b-4cb5-a9c1-dd078e83b2f1-kube-api-access-tgf79\") pod \"aws-ebs-csi-driver-node-hf9sf\" (UID: \"dc814136-ef4b-4cb5-a9c1-dd078e83b2f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf9sf" Jul 14 17:25:57.464561 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.464518 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52svk\" (UniqueName: \"kubernetes.io/projected/d2bc074a-026c-402d-ac70-7d39ff8dc81f-kube-api-access-52svk\") pod \"iptables-alerter-c7d6p\" (UID: \"d2bc074a-026c-402d-ac70-7d39ff8dc81f\") " pod="openshift-network-operator/iptables-alerter-c7d6p" Jul 14 17:25:57.465400 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.465375 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzqfd\" (UniqueName: \"kubernetes.io/projected/69392349-f201-43dd-9748-78613041a9a9-kube-api-access-bzqfd\") pod \"tuned-pqnvp\" (UID: \"69392349-f201-43dd-9748-78613041a9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.465827 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.465803 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsxs8\" (UniqueName: \"kubernetes.io/projected/b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb-kube-api-access-qsxs8\") pod \"network-metrics-daemon-ctxxr\" (UID: \"b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb\") " pod="openshift-multus/network-metrics-daemon-ctxxr" Jul 14 17:25:57.466509 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.466486 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25t4c\" (UniqueName: \"kubernetes.io/projected/814adb62-03ce-494d-9e1f-1b78f45b87e2-kube-api-access-25t4c\") pod \"multus-additional-cni-plugins-k9kwq\" (UID: \"814adb62-03ce-494d-9e1f-1b78f45b87e2\") " pod="openshift-multus/multus-additional-cni-plugins-k9kwq" Jul 14 17:25:57.468181 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.468160 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwrkj\" (UniqueName: \"kubernetes.io/projected/7f7817bc-7352-4f6a-9f20-cf101d130a7c-kube-api-access-xwrkj\") pod \"node-ca-7ffq4\" (UID: \"7f7817bc-7352-4f6a-9f20-cf101d130a7c\") " pod="openshift-image-registry/node-ca-7ffq4" Jul 14 17:25:57.468441 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.468416 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2k62s\" (UniqueName: \"kubernetes.io/projected/7768649b-c82e-4a04-a4c1-93061292bcec-kube-api-access-2k62s\") pod \"node-resolver-b8gnk\" (UID: \"7768649b-c82e-4a04-a4c1-93061292bcec\") " pod="openshift-dns/node-resolver-b8gnk" Jul 14 17:25:57.468870 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.468852 2554 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 17:25:57.469024 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.468958 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gktpm\" (UniqueName: \"kubernetes.io/projected/a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a-kube-api-access-gktpm\") pod \"multus-ct2sx\" (UID: \"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a\") " pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.531042 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.531019 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-b8gnk" Jul 14 17:25:57.543110 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.543078 2554 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 17:25:57.545161 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.545140 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7ffq4" Jul 14 17:25:57.546380 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:57.546354 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93bf474703caedf9ae57eedb84b07448.slice/crio-e1042c8af75f1109b6b23f35ac7cefa780e2ad29da27f8e0c12b156bb65fd1e1 WatchSource:0}: Error finding container e1042c8af75f1109b6b23f35ac7cefa780e2ad29da27f8e0c12b156bb65fd1e1: Status 404 returned error can't find the container with id e1042c8af75f1109b6b23f35ac7cefa780e2ad29da27f8e0c12b156bb65fd1e1 Jul 14 17:25:57.547345 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:57.547318 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7768649b_c82e_4a04_a4c1_93061292bcec.slice/crio-e5199871993f23ff07210fd0f6718c7372f7e9e89c6fd98c7c028ccc54e9409a WatchSource:0}: Error finding container e5199871993f23ff07210fd0f6718c7372f7e9e89c6fd98c7c028ccc54e9409a: Status 404 returned error can't find the container with id e5199871993f23ff07210fd0f6718c7372f7e9e89c6fd98c7c028ccc54e9409a Jul 14 17:25:57.547677 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:57.547657 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3ea3cd1054689f0ec1aa0dca0048bb1.slice/crio-7c8a56cd08b54fb5519f119b03148c775179be6bd193de502af3cdce939cc7fd WatchSource:0}: Error finding container 7c8a56cd08b54fb5519f119b03148c775179be6bd193de502af3cdce939cc7fd: Status 404 returned error can't find the container with id 7c8a56cd08b54fb5519f119b03148c775179be6bd193de502af3cdce939cc7fd Jul 14 17:25:57.553167 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.553005 2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 17:25:57.556391 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:57.556366 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f7817bc_7352_4f6a_9f20_cf101d130a7c.slice/crio-0cc0d575e5c8974a698ec15ba871d911aaf3764d1daed68fe03abbdce6a207b4 WatchSource:0}: Error finding container 0cc0d575e5c8974a698ec15ba871d911aaf3764d1daed68fe03abbdce6a207b4: Status 404 returned error can't find the container with id 0cc0d575e5c8974a698ec15ba871d911aaf3764d1daed68fe03abbdce6a207b4 Jul 14 17:25:57.564448 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.564418 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-k9kwq" Jul 14 17:25:57.576057 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:57.576034 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod814adb62_03ce_494d_9e1f_1b78f45b87e2.slice/crio-135778cb2e20b71076b2a3bab831558366acbea94d80d6e79ce8d3ad8f6e8d04 WatchSource:0}: Error finding container 135778cb2e20b71076b2a3bab831558366acbea94d80d6e79ce8d3ad8f6e8d04: Status 404 returned error can't find the container with id 135778cb2e20b71076b2a3bab831558366acbea94d80d6e79ce8d3ad8f6e8d04 Jul 14 17:25:57.577734 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.577716 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:25:57.585113 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.585089 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-djms6" Jul 14 17:25:57.585303 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:57.585267 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3f90a0c_1b3b_415f_9ed6_84bcbdd6a726.slice/crio-4f32f2be76b61df2c1278323683b38390bdc7927e047d4af24d47dd08bec2926 WatchSource:0}: Error finding container 4f32f2be76b61df2c1278323683b38390bdc7927e047d4af24d47dd08bec2926: Status 404 returned error can't find the container with id 4f32f2be76b61df2c1278323683b38390bdc7927e047d4af24d47dd08bec2926 Jul 14 17:25:57.593359 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:57.593339 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod210c935e_f5d4_4de0_abee_7d5e2f555f34.slice/crio-2a6e373305e6878ea92cdf32df5964311db92130d5c0e23857e372e170da0eb0 WatchSource:0}: Error finding container 2a6e373305e6878ea92cdf32df5964311db92130d5c0e23857e372e170da0eb0: Status 404 returned error can't find the container with id 2a6e373305e6878ea92cdf32df5964311db92130d5c0e23857e372e170da0eb0 Jul 14 17:25:57.612822 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.612788 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" Jul 14 17:25:57.620916 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:57.620892 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69392349_f201_43dd_9748_78613041a9a9.slice/crio-29c2b32531cf02efc287500af7250e42b933e7346dec2d34c50620d04e2a77e5 WatchSource:0}: Error finding container 29c2b32531cf02efc287500af7250e42b933e7346dec2d34c50620d04e2a77e5: Status 404 returned error can't find the container with id 29c2b32531cf02efc287500af7250e42b933e7346dec2d34c50620d04e2a77e5 Jul 14 17:25:57.633315 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.633297 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ct2sx" Jul 14 17:25:57.641510 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:57.641490 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4c3f7c7_51a3_43de_9e9f_b2e26cc9985a.slice/crio-31b8f32fe7bed833c13f49fe2bd35764047fea5e55b75f788feaeef01f24d2a3 WatchSource:0}: Error finding container 31b8f32fe7bed833c13f49fe2bd35764047fea5e55b75f788feaeef01f24d2a3: Status 404 returned error can't find the container with id 31b8f32fe7bed833c13f49fe2bd35764047fea5e55b75f788feaeef01f24d2a3 Jul 14 17:25:57.653566 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.653549 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-c7d6p" Jul 14 17:25:57.657549 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.657453 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf9sf" Jul 14 17:25:57.662792 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:57.662766 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2bc074a_026c_402d_ac70_7d39ff8dc81f.slice/crio-d0641bdd63fa6597c706a8417601ab551fadad24264d75612a73e8e9bddef016 WatchSource:0}: Error finding container d0641bdd63fa6597c706a8417601ab551fadad24264d75612a73e8e9bddef016: Status 404 returned error can't find the container with id d0641bdd63fa6597c706a8417601ab551fadad24264d75612a73e8e9bddef016 Jul 14 17:25:57.666851 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:25:57.666831 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc814136_ef4b_4cb5_a9c1_dd078e83b2f1.slice/crio-dae14115700500da780a883ffd03714239d3172e9ec4f692f73ff08c5e53ccd1 WatchSource:0}: Error finding container dae14115700500da780a883ffd03714239d3172e9ec4f692f73ff08c5e53ccd1: Status 404 returned error can't find the container with id dae14115700500da780a883ffd03714239d3172e9ec4f692f73ff08c5e53ccd1 Jul 14 17:25:57.668455 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:57.668437 2554 kuberuntime_manager.go:1459] "Unhandled Error" err="container csi-driver start failed in pod aws-ebs-csi-driver-node-hf9sf_openshift-cluster-csi-drivers(dc814136-ef4b-4cb5-a9c1-dd078e83b2f1): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Jul 14 17:25:57.669203 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:57.669182 2554 kuberuntime_manager.go:1459] "Unhandled Error" err="container csi-node-driver-registrar start failed in pod aws-ebs-csi-driver-node-hf9sf_openshift-cluster-csi-drivers(dc814136-ef4b-4cb5-a9c1-dd078e83b2f1): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Jul 14 17:25:57.670918 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:57.670901 2554 kuberuntime_manager.go:1459] "Unhandled Error" err="container csi-liveness-probe start failed in pod aws-ebs-csi-driver-node-hf9sf_openshift-cluster-csi-drivers(dc814136-ef4b-4cb5-a9c1-dd078e83b2f1): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Jul 14 17:25:57.670962 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:57.670937 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"csi-driver\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"csi-node-driver-registrar\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"csi-liveness-probe\" with ErrImagePull: \"pull QPS exceeded\"]" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf9sf" podUID="dc814136-ef4b-4cb5-a9c1-dd078e83b2f1" Jul 14 17:25:57.959467 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.959379 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb-metrics-certs\") pod \"network-metrics-daemon-ctxxr\" (UID: \"b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb\") " pod="openshift-multus/network-metrics-daemon-ctxxr" Jul 14 17:25:57.959467 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:57.959442 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69371a04-a55d-4548-a7df-2868cd880309-original-pull-secret\") pod \"global-pull-secret-syncer-88sg9\" (UID: \"69371a04-a55d-4548-a7df-2868cd880309\") " pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:25:57.959663 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:57.959527 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 17:25:57.959663 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:57.959529 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:25:57.959663 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:57.959577 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69371a04-a55d-4548-a7df-2868cd880309-original-pull-secret podName:69371a04-a55d-4548-a7df-2868cd880309 nodeName:}" failed. No retries permitted until 2026-07-14 17:25:58.959564197 +0000 UTC m=+3.100812881 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/69371a04-a55d-4548-a7df-2868cd880309-original-pull-secret") pod "global-pull-secret-syncer-88sg9" (UID: "69371a04-a55d-4548-a7df-2868cd880309") : object "kube-system"/"original-pull-secret" not registered Jul 14 17:25:57.959663 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:57.959591 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb-metrics-certs podName:b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb nodeName:}" failed. No retries permitted until 2026-07-14 17:25:58.959584588 +0000 UTC m=+3.100833272 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb-metrics-certs") pod "network-metrics-daemon-ctxxr" (UID: "b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:25:58.060547 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:58.060513 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq99q\" (UniqueName: \"kubernetes.io/projected/56fc04ea-db5c-4d02-a7ff-a92cf718cb15-kube-api-access-nq99q\") pod \"network-check-target-4lhqc\" (UID: \"56fc04ea-db5c-4d02-a7ff-a92cf718cb15\") " pod="openshift-network-diagnostics/network-check-target-4lhqc" Jul 14 17:25:58.060781 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:58.060701 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 17:25:58.060781 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:58.060731 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 17:25:58.060781 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:58.060745 2554 projected.go:196] Error preparing data for projected volume kube-api-access-nq99q for pod openshift-network-diagnostics/network-check-target-4lhqc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:25:58.060956 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:58.060811 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/56fc04ea-db5c-4d02-a7ff-a92cf718cb15-kube-api-access-nq99q podName:56fc04ea-db5c-4d02-a7ff-a92cf718cb15 nodeName:}" failed. No retries permitted until 2026-07-14 17:25:59.060788821 +0000 UTC m=+3.202037527 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-nq99q" (UniqueName: "kubernetes.io/projected/56fc04ea-db5c-4d02-a7ff-a92cf718cb15-kube-api-access-nq99q") pod "network-check-target-4lhqc" (UID: "56fc04ea-db5c-4d02-a7ff-a92cf718cb15") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:25:58.102663 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:58.102634 2554 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 17:25:58.250152 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:58.250000 2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 17:20:57 +0000 UTC" deadline="2028-04-22 08:03:56.375038808 +0000 UTC" Jul 14 17:25:58.250152 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:58.250119 2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15542h37m58.124925328s" Jul 14 17:25:58.420133 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:58.420046 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-b8gnk" event={"ID":"7768649b-c82e-4a04-a4c1-93061292bcec","Type":"ContainerStarted","Data":"e5199871993f23ff07210fd0f6718c7372f7e9e89c6fd98c7c028ccc54e9409a"} Jul 14 17:25:58.426078 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:58.425781 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-13.ec2.internal" event={"ID":"93bf474703caedf9ae57eedb84b07448","Type":"ContainerStarted","Data":"e1042c8af75f1109b6b23f35ac7cefa780e2ad29da27f8e0c12b156bb65fd1e1"} Jul 14 17:25:58.439273 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:58.439240 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" event={"ID":"69392349-f201-43dd-9748-78613041a9a9","Type":"ContainerStarted","Data":"29c2b32531cf02efc287500af7250e42b933e7346dec2d34c50620d04e2a77e5"} Jul 14 17:25:58.447015 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:58.446949 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-djms6" event={"ID":"210c935e-f5d4-4de0-abee-7d5e2f555f34","Type":"ContainerStarted","Data":"2a6e373305e6878ea92cdf32df5964311db92130d5c0e23857e372e170da0eb0"} Jul 14 17:25:58.458055 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:58.458028 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k9kwq" event={"ID":"814adb62-03ce-494d-9e1f-1b78f45b87e2","Type":"ContainerStarted","Data":"135778cb2e20b71076b2a3bab831558366acbea94d80d6e79ce8d3ad8f6e8d04"} Jul 14 17:25:58.466553 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:58.466523 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7ffq4" event={"ID":"7f7817bc-7352-4f6a-9f20-cf101d130a7c","Type":"ContainerStarted","Data":"0cc0d575e5c8974a698ec15ba871d911aaf3764d1daed68fe03abbdce6a207b4"} Jul 14 17:25:58.468303 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:58.468270 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-13.ec2.internal" event={"ID":"b3ea3cd1054689f0ec1aa0dca0048bb1","Type":"ContainerStarted","Data":"7c8a56cd08b54fb5519f119b03148c775179be6bd193de502af3cdce939cc7fd"} Jul 14 17:25:58.490522 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:58.490491 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf9sf" event={"ID":"dc814136-ef4b-4cb5-a9c1-dd078e83b2f1","Type":"ContainerStarted","Data":"dae14115700500da780a883ffd03714239d3172e9ec4f692f73ff08c5e53ccd1"} Jul 14 17:25:58.508592 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:58.508552 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"csi-driver\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:23e38a7cf7c4e5c572a5461047f39d2f821e55490088f442a842f66da801ae99\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"csi-node-driver-registrar\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0512f4cb34792d3c7553c42a85869c34b5953550c019b958649e3246d6540d94\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"csi-liveness-probe\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0f984e9f25655116e61604ac13881b82e2830789119e02b44a338cea95f30425\\\": ErrImagePull: pull QPS exceeded\"]" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf9sf" podUID="dc814136-ef4b-4cb5-a9c1-dd078e83b2f1" Jul 14 17:25:58.511190 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:58.511162 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-c7d6p" event={"ID":"d2bc074a-026c-402d-ac70-7d39ff8dc81f","Type":"ContainerStarted","Data":"d0641bdd63fa6597c706a8417601ab551fadad24264d75612a73e8e9bddef016"} Jul 14 17:25:58.534493 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:58.534463 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ct2sx" event={"ID":"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a","Type":"ContainerStarted","Data":"31b8f32fe7bed833c13f49fe2bd35764047fea5e55b75f788feaeef01f24d2a3"} Jul 14 17:25:58.543705 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:58.543337 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" event={"ID":"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726","Type":"ContainerStarted","Data":"4f32f2be76b61df2c1278323683b38390bdc7927e047d4af24d47dd08bec2926"} Jul 14 17:25:58.967788 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:58.967708 2554 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 17:25:58.968262 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:58.968225 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69371a04-a55d-4548-a7df-2868cd880309-original-pull-secret\") pod \"global-pull-secret-syncer-88sg9\" (UID: \"69371a04-a55d-4548-a7df-2868cd880309\") " pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:25:58.968408 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:58.968302 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb-metrics-certs\") pod \"network-metrics-daemon-ctxxr\" (UID: \"b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb\") " pod="openshift-multus/network-metrics-daemon-ctxxr" Jul 14 17:25:58.968471 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:58.968423 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 17:25:58.968471 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:58.968456 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:25:58.968569 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:58.968507 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69371a04-a55d-4548-a7df-2868cd880309-original-pull-secret podName:69371a04-a55d-4548-a7df-2868cd880309 nodeName:}" failed. No retries permitted until 2026-07-14 17:26:00.968483312 +0000 UTC m=+5.109732025 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/69371a04-a55d-4548-a7df-2868cd880309-original-pull-secret") pod "global-pull-secret-syncer-88sg9" (UID: "69371a04-a55d-4548-a7df-2868cd880309") : object "kube-system"/"original-pull-secret" not registered Jul 14 17:25:58.968569 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:58.968529 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb-metrics-certs podName:b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb nodeName:}" failed. No retries permitted until 2026-07-14 17:26:00.968517286 +0000 UTC m=+5.109765974 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb-metrics-certs") pod "network-metrics-daemon-ctxxr" (UID: "b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:25:59.069616 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:59.069572 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq99q\" (UniqueName: \"kubernetes.io/projected/56fc04ea-db5c-4d02-a7ff-a92cf718cb15-kube-api-access-nq99q\") pod \"network-check-target-4lhqc\" (UID: \"56fc04ea-db5c-4d02-a7ff-a92cf718cb15\") " pod="openshift-network-diagnostics/network-check-target-4lhqc" Jul 14 17:25:59.069808 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:59.069784 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 17:25:59.069808 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:59.069806 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 17:25:59.069926 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:59.069818 2554 projected.go:196] Error preparing data for projected volume kube-api-access-nq99q for pod openshift-network-diagnostics/network-check-target-4lhqc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:25:59.069926 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:59.069878 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/56fc04ea-db5c-4d02-a7ff-a92cf718cb15-kube-api-access-nq99q podName:56fc04ea-db5c-4d02-a7ff-a92cf718cb15 nodeName:}" failed. No retries permitted until 2026-07-14 17:26:01.069858751 +0000 UTC m=+5.211107448 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-nq99q" (UniqueName: "kubernetes.io/projected/56fc04ea-db5c-4d02-a7ff-a92cf718cb15-kube-api-access-nq99q") pod "network-check-target-4lhqc" (UID: "56fc04ea-db5c-4d02-a7ff-a92cf718cb15") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:25:59.250771 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:59.250653 2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 17:20:57 +0000 UTC" deadline="2028-01-02 07:33:36.865221878 +0000 UTC" Jul 14 17:25:59.250771 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:59.250719 2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12878h7m37.61450722s" Jul 14 17:25:59.382662 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:59.382632 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4lhqc" Jul 14 17:25:59.382852 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:59.382765 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4lhqc" podUID="56fc04ea-db5c-4d02-a7ff-a92cf718cb15" Jul 14 17:25:59.383240 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:59.383196 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:25:59.383434 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:59.383410 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88sg9" podUID="69371a04-a55d-4548-a7df-2868cd880309" Jul 14 17:25:59.383511 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:25:59.383488 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctxxr" Jul 14 17:25:59.383625 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:25:59.383586 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctxxr" podUID="b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb" Jul 14 17:26:00.988730 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:00.988640 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69371a04-a55d-4548-a7df-2868cd880309-original-pull-secret\") pod \"global-pull-secret-syncer-88sg9\" (UID: \"69371a04-a55d-4548-a7df-2868cd880309\") " pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:26:00.988730 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:00.988708 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb-metrics-certs\") pod \"network-metrics-daemon-ctxxr\" (UID: \"b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb\") " pod="openshift-multus/network-metrics-daemon-ctxxr" Jul 14 17:26:00.989269 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:00.988848 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:26:00.989269 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:00.988923 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb-metrics-certs podName:b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb nodeName:}" failed. No retries permitted until 2026-07-14 17:26:04.988900997 +0000 UTC m=+9.130149705 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb-metrics-certs") pod "network-metrics-daemon-ctxxr" (UID: "b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:26:00.989445 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:00.989401 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 17:26:00.989504 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:00.989463 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69371a04-a55d-4548-a7df-2868cd880309-original-pull-secret podName:69371a04-a55d-4548-a7df-2868cd880309 nodeName:}" failed. No retries permitted until 2026-07-14 17:26:04.989446127 +0000 UTC m=+9.130694814 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/69371a04-a55d-4548-a7df-2868cd880309-original-pull-secret") pod "global-pull-secret-syncer-88sg9" (UID: "69371a04-a55d-4548-a7df-2868cd880309") : object "kube-system"/"original-pull-secret" not registered Jul 14 17:26:01.090100 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:01.089443 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq99q\" (UniqueName: \"kubernetes.io/projected/56fc04ea-db5c-4d02-a7ff-a92cf718cb15-kube-api-access-nq99q\") pod \"network-check-target-4lhqc\" (UID: \"56fc04ea-db5c-4d02-a7ff-a92cf718cb15\") " pod="openshift-network-diagnostics/network-check-target-4lhqc" Jul 14 17:26:01.090100 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:01.089628 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 17:26:01.090100 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:01.089661 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 17:26:01.090100 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:01.089676 2554 projected.go:196] Error preparing data for projected volume kube-api-access-nq99q for pod openshift-network-diagnostics/network-check-target-4lhqc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:26:01.090100 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:01.089742 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/56fc04ea-db5c-4d02-a7ff-a92cf718cb15-kube-api-access-nq99q podName:56fc04ea-db5c-4d02-a7ff-a92cf718cb15 nodeName:}" failed. No retries permitted until 2026-07-14 17:26:05.089721705 +0000 UTC m=+9.230970397 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-nq99q" (UniqueName: "kubernetes.io/projected/56fc04ea-db5c-4d02-a7ff-a92cf718cb15-kube-api-access-nq99q") pod "network-check-target-4lhqc" (UID: "56fc04ea-db5c-4d02-a7ff-a92cf718cb15") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:26:01.382069 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:01.381831 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:26:01.382069 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:01.381983 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88sg9" podUID="69371a04-a55d-4548-a7df-2868cd880309" Jul 14 17:26:01.382333 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:01.382162 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctxxr" Jul 14 17:26:01.382396 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:01.382329 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctxxr" podUID="b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb" Jul 14 17:26:01.382396 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:01.382389 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4lhqc" Jul 14 17:26:01.382547 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:01.382492 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4lhqc" podUID="56fc04ea-db5c-4d02-a7ff-a92cf718cb15" Jul 14 17:26:03.382156 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:03.381756 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4lhqc" Jul 14 17:26:03.382156 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:03.381756 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctxxr" Jul 14 17:26:03.382156 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:03.381775 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:26:03.382156 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:03.381914 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4lhqc" podUID="56fc04ea-db5c-4d02-a7ff-a92cf718cb15" Jul 14 17:26:03.382156 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:03.381980 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88sg9" podUID="69371a04-a55d-4548-a7df-2868cd880309" Jul 14 17:26:03.382156 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:03.382081 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctxxr" podUID="b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb" Jul 14 17:26:05.025096 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:05.025055 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69371a04-a55d-4548-a7df-2868cd880309-original-pull-secret\") pod \"global-pull-secret-syncer-88sg9\" (UID: \"69371a04-a55d-4548-a7df-2868cd880309\") " pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:26:05.025675 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:05.025178 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb-metrics-certs\") pod \"network-metrics-daemon-ctxxr\" (UID: \"b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb\") " pod="openshift-multus/network-metrics-daemon-ctxxr" Jul 14 17:26:05.025675 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:05.025237 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 17:26:05.025675 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:05.025314 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69371a04-a55d-4548-a7df-2868cd880309-original-pull-secret podName:69371a04-a55d-4548-a7df-2868cd880309 nodeName:}" failed. No retries permitted until 2026-07-14 17:26:13.025291184 +0000 UTC m=+17.166539876 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/69371a04-a55d-4548-a7df-2868cd880309-original-pull-secret") pod "global-pull-secret-syncer-88sg9" (UID: "69371a04-a55d-4548-a7df-2868cd880309") : object "kube-system"/"original-pull-secret" not registered Jul 14 17:26:05.025675 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:05.025410 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:26:05.025675 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:05.025466 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb-metrics-certs podName:b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb nodeName:}" failed. No retries permitted until 2026-07-14 17:26:13.025448154 +0000 UTC m=+17.166696854 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb-metrics-certs") pod "network-metrics-daemon-ctxxr" (UID: "b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:26:05.125700 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:05.125669 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq99q\" (UniqueName: \"kubernetes.io/projected/56fc04ea-db5c-4d02-a7ff-a92cf718cb15-kube-api-access-nq99q\") pod \"network-check-target-4lhqc\" (UID: \"56fc04ea-db5c-4d02-a7ff-a92cf718cb15\") " pod="openshift-network-diagnostics/network-check-target-4lhqc" Jul 14 17:26:05.125912 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:05.125858 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 17:26:05.125912 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:05.125878 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 17:26:05.125912 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:05.125890 2554 projected.go:196] Error preparing data for projected volume kube-api-access-nq99q for pod openshift-network-diagnostics/network-check-target-4lhqc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:26:05.126078 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:05.125958 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/56fc04ea-db5c-4d02-a7ff-a92cf718cb15-kube-api-access-nq99q podName:56fc04ea-db5c-4d02-a7ff-a92cf718cb15 nodeName:}" failed. No retries permitted until 2026-07-14 17:26:13.125939606 +0000 UTC m=+17.267188291 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-nq99q" (UniqueName: "kubernetes.io/projected/56fc04ea-db5c-4d02-a7ff-a92cf718cb15-kube-api-access-nq99q") pod "network-check-target-4lhqc" (UID: "56fc04ea-db5c-4d02-a7ff-a92cf718cb15") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:26:05.382854 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:05.382484 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4lhqc" Jul 14 17:26:05.382854 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:05.382532 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:26:05.382854 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:05.382632 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4lhqc" podUID="56fc04ea-db5c-4d02-a7ff-a92cf718cb15" Jul 14 17:26:05.382854 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:05.382718 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctxxr" Jul 14 17:26:05.382854 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:05.382828 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctxxr" podUID="b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb" Jul 14 17:26:05.383244 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:05.383058 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88sg9" podUID="69371a04-a55d-4548-a7df-2868cd880309" Jul 14 17:26:07.382682 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:07.382645 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:26:07.383130 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:07.382767 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctxxr" Jul 14 17:26:07.383130 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:07.382780 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88sg9" podUID="69371a04-a55d-4548-a7df-2868cd880309" Jul 14 17:26:07.383130 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:07.382896 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctxxr" podUID="b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb" Jul 14 17:26:07.383130 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:07.382961 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4lhqc" Jul 14 17:26:07.383130 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:07.383028 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4lhqc" podUID="56fc04ea-db5c-4d02-a7ff-a92cf718cb15" Jul 14 17:26:09.382414 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:09.382113 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4lhqc" Jul 14 17:26:09.382414 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:09.382144 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:26:09.382858 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:09.382174 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctxxr" Jul 14 17:26:09.382858 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:09.382472 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4lhqc" podUID="56fc04ea-db5c-4d02-a7ff-a92cf718cb15" Jul 14 17:26:09.382858 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:09.382541 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88sg9" podUID="69371a04-a55d-4548-a7df-2868cd880309" Jul 14 17:26:09.382858 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:09.382629 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctxxr" podUID="b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb" Jul 14 17:26:11.382047 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:11.382014 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:26:11.382498 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:11.382019 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4lhqc" Jul 14 17:26:11.382498 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:11.382163 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctxxr" Jul 14 17:26:11.382498 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:11.382160 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88sg9" podUID="69371a04-a55d-4548-a7df-2868cd880309" Jul 14 17:26:11.382498 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:11.382250 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4lhqc" podUID="56fc04ea-db5c-4d02-a7ff-a92cf718cb15" Jul 14 17:26:11.382498 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:11.382335 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctxxr" podUID="b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb" Jul 14 17:26:13.085584 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:13.085542 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb-metrics-certs\") pod \"network-metrics-daemon-ctxxr\" (UID: \"b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb\") " pod="openshift-multus/network-metrics-daemon-ctxxr" Jul 14 17:26:13.085990 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:13.085620 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69371a04-a55d-4548-a7df-2868cd880309-original-pull-secret\") pod \"global-pull-secret-syncer-88sg9\" (UID: \"69371a04-a55d-4548-a7df-2868cd880309\") " pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:26:13.085990 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:13.085703 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 17:26:13.085990 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:13.085762 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69371a04-a55d-4548-a7df-2868cd880309-original-pull-secret podName:69371a04-a55d-4548-a7df-2868cd880309 nodeName:}" failed. No retries permitted until 2026-07-14 17:26:29.085749574 +0000 UTC m=+33.226998256 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/69371a04-a55d-4548-a7df-2868cd880309-original-pull-secret") pod "global-pull-secret-syncer-88sg9" (UID: "69371a04-a55d-4548-a7df-2868cd880309") : object "kube-system"/"original-pull-secret" not registered Jul 14 17:26:13.085990 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:13.085702 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:26:13.085990 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:13.085843 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb-metrics-certs podName:b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb nodeName:}" failed. No retries permitted until 2026-07-14 17:26:29.085828179 +0000 UTC m=+33.227076865 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb-metrics-certs") pod "network-metrics-daemon-ctxxr" (UID: "b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:26:13.186784 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:13.186753 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq99q\" (UniqueName: \"kubernetes.io/projected/56fc04ea-db5c-4d02-a7ff-a92cf718cb15-kube-api-access-nq99q\") pod \"network-check-target-4lhqc\" (UID: \"56fc04ea-db5c-4d02-a7ff-a92cf718cb15\") " pod="openshift-network-diagnostics/network-check-target-4lhqc" Jul 14 17:26:13.186970 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:13.186956 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 17:26:13.187025 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:13.186981 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 17:26:13.187025 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:13.186996 2554 projected.go:196] Error preparing data for projected volume kube-api-access-nq99q for pod openshift-network-diagnostics/network-check-target-4lhqc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:26:13.187116 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:13.187059 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/56fc04ea-db5c-4d02-a7ff-a92cf718cb15-kube-api-access-nq99q podName:56fc04ea-db5c-4d02-a7ff-a92cf718cb15 nodeName:}" failed. No retries permitted until 2026-07-14 17:26:29.187041202 +0000 UTC m=+33.328289898 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-nq99q" (UniqueName: "kubernetes.io/projected/56fc04ea-db5c-4d02-a7ff-a92cf718cb15-kube-api-access-nq99q") pod "network-check-target-4lhqc" (UID: "56fc04ea-db5c-4d02-a7ff-a92cf718cb15") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:26:13.382576 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:13.382474 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:26:13.382576 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:13.382474 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4lhqc" Jul 14 17:26:13.382576 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:13.382542 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctxxr" Jul 14 17:26:13.382894 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:13.382850 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4lhqc" podUID="56fc04ea-db5c-4d02-a7ff-a92cf718cb15" Jul 14 17:26:13.383019 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:13.382906 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctxxr" podUID="b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb" Jul 14 17:26:13.383019 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:13.382999 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88sg9" podUID="69371a04-a55d-4548-a7df-2868cd880309" Jul 14 17:26:15.381685 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:15.381648 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4lhqc" Jul 14 17:26:15.382105 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:15.381774 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4lhqc" podUID="56fc04ea-db5c-4d02-a7ff-a92cf718cb15" Jul 14 17:26:15.382105 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:15.381650 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctxxr" Jul 14 17:26:15.382105 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:15.381648 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:26:15.382105 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:15.381926 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctxxr" podUID="b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb" Jul 14 17:26:15.382105 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:15.381997 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88sg9" podUID="69371a04-a55d-4548-a7df-2868cd880309" Jul 14 17:26:17.382650 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:17.382618 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctxxr" Jul 14 17:26:17.382650 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:17.382641 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:26:17.383033 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:17.382737 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4lhqc" Jul 14 17:26:17.383033 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:17.382752 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctxxr" podUID="b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb" Jul 14 17:26:17.383033 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:17.382827 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4lhqc" podUID="56fc04ea-db5c-4d02-a7ff-a92cf718cb15" Jul 14 17:26:17.383033 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:17.382921 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88sg9" podUID="69371a04-a55d-4548-a7df-2868cd880309" Jul 14 17:26:17.607931 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:17.607847 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ct2sx" event={"ID":"a4c3f7c7-51a3-43de-9e9f-b2e26cc9985a","Type":"ContainerStarted","Data":"2b8cbf4ae6baf6c12a2f11dd62b5010e8118288ea6cb4cf39420c9605d08d07b"} Jul 14 17:26:17.616042 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:17.616015 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" event={"ID":"69392349-f201-43dd-9748-78613041a9a9","Type":"ContainerStarted","Data":"d744d464e8a5adc0e337b6c2d206c2041dc31f96b0940c437f4e7d3fee442bd6"} Jul 14 17:26:17.623306 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:17.623273 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-13.ec2.internal" event={"ID":"b3ea3cd1054689f0ec1aa0dca0048bb1","Type":"ContainerStarted","Data":"39769e1c79bd998c0b501f1e5a2b05f3db52f53e5948c62ba9499e89146326ca"} Jul 14 17:26:17.625162 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:17.624859 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-ct2sx" podStartSLOduration=0.947341276 podStartE2EDuration="20.624841391s" totalImagesPullingTime="19.677500115s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:25:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:26:17.624421128 +0000 UTC m=+21.765669833" watchObservedRunningTime="2026-07-14 17:26:17.624841391 +0000 UTC m=+21.766090096" Jul 14 17:26:17.639323 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:17.638917 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-13.ec2.internal" podStartSLOduration=20.638897252 podStartE2EDuration="20.638897252s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:25:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:26:17.638434809 +0000 UTC m=+21.779683514" watchObservedRunningTime="2026-07-14 17:26:17.638897252 +0000 UTC m=+21.780145972" Jul 14 17:26:17.654724 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:17.654668 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-pqnvp" podStartSLOduration=1.043132673 podStartE2EDuration="20.654653659s" totalImagesPullingTime="19.611520986s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:25:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:26:17.654638915 +0000 UTC m=+21.795887619" watchObservedRunningTime="2026-07-14 17:26:17.654653659 +0000 UTC m=+21.795902365" Jul 14 17:26:18.626927 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:18.626645 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-b8gnk" event={"ID":"7768649b-c82e-4a04-a4c1-93061292bcec","Type":"ContainerStarted","Data":"65d7d643afd76bed3f3ebbeb06419b02c69f669a0530bc81d338db77d938fc15"} Jul 14 17:26:18.628296 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:18.628270 2554 generic.go:356] "Generic (PLEG): container finished" podID="93bf474703caedf9ae57eedb84b07448" containerID="57da0580fe4ad793943dec19911f9751969cdf0eddea9d6d4091b709e99415d0" exitCode=0 Jul 14 17:26:18.628403 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:18.628338 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-13.ec2.internal" event={"ID":"93bf474703caedf9ae57eedb84b07448","Type":"ContainerDied","Data":"57da0580fe4ad793943dec19911f9751969cdf0eddea9d6d4091b709e99415d0"} Jul 14 17:26:18.629983 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:18.629955 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-djms6" event={"ID":"210c935e-f5d4-4de0-abee-7d5e2f555f34","Type":"ContainerStarted","Data":"dbbfdac8ba1f8dbaf010ad3b14e0e14269a877e7196ef0e69ade06c482b92b78"} Jul 14 17:26:18.631473 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:18.631452 2554 generic.go:356] "Generic (PLEG): container finished" podID="814adb62-03ce-494d-9e1f-1b78f45b87e2" containerID="7d6a30fb03106219cc5079e516cc545fd72563ddd04caf7d883c7a124aaf2da4" exitCode=0 Jul 14 17:26:18.631611 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:18.631535 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k9kwq" event={"ID":"814adb62-03ce-494d-9e1f-1b78f45b87e2","Type":"ContainerDied","Data":"7d6a30fb03106219cc5079e516cc545fd72563ddd04caf7d883c7a124aaf2da4"} Jul 14 17:26:18.633291 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:18.633263 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7ffq4" event={"ID":"7f7817bc-7352-4f6a-9f20-cf101d130a7c","Type":"ContainerStarted","Data":"6eb080b4c645a2d8770bf69e841087bb3f65c544ae58ec25e9e1ff58d5cf47c9"} Jul 14 17:26:18.635604 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:18.635584 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pk9h2_f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726/ovn-acl-logging/0.log" Jul 14 17:26:18.635974 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:18.635954 2554 generic.go:356] "Generic (PLEG): container finished" podID="f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726" containerID="159ac3e05c8580462cdb692a64576922a5f09efab59f4b1d45ebfe65210aaeb3" exitCode=1 Jul 14 17:26:18.636059 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:18.636045 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" event={"ID":"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726","Type":"ContainerStarted","Data":"c0112840640293b7ea32e200b8f58f8ffc6fbf2b4bf91a121436ae80f3e0ed0f"} Jul 14 17:26:18.636133 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:18.636116 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" event={"ID":"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726","Type":"ContainerStarted","Data":"6f44b0f2fb1188199632f895223c81570c2df0593a5fcc9ca50613e460ff8ff1"} Jul 14 17:26:18.636188 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:18.636174 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" event={"ID":"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726","Type":"ContainerDied","Data":"159ac3e05c8580462cdb692a64576922a5f09efab59f4b1d45ebfe65210aaeb3"} Jul 14 17:26:18.636259 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:18.636228 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" event={"ID":"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726","Type":"ContainerStarted","Data":"fa5f8a3e4fe26b5f9e5176d9777fb5aeac3efe8bc181d3632dc12bec8a0e7b67"} Jul 14 17:26:18.643745 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:18.643702 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-b8gnk" podStartSLOduration=1.885706553 podStartE2EDuration="21.643689239s" totalImagesPullingTime="19.757982686s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:25:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:26:18.643387259 +0000 UTC m=+22.784635964" watchObservedRunningTime="2026-07-14 17:26:18.643689239 +0000 UTC m=+22.784937943" Jul 14 17:26:18.673419 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:18.673374 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7ffq4" podStartSLOduration=1.9979775210000001 podStartE2EDuration="21.673359227s" totalImagesPullingTime="19.675381706s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:25:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:26:18.67294423 +0000 UTC m=+22.814192946" watchObservedRunningTime="2026-07-14 17:26:18.673359227 +0000 UTC m=+22.814607932" Jul 14 17:26:18.734522 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:18.734475 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-djms6" podStartSLOduration=2.095195749 podStartE2EDuration="21.7344569s" totalImagesPullingTime="19.639261151s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:25:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:26:18.734432264 +0000 UTC m=+22.875680968" watchObservedRunningTime="2026-07-14 17:26:18.7344569 +0000 UTC m=+22.875705605" Jul 14 17:26:19.381944 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:19.381868 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4lhqc" Jul 14 17:26:19.381944 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:19.381922 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:26:19.382306 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:19.382029 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctxxr" Jul 14 17:26:19.382306 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:19.382045 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4lhqc" podUID="56fc04ea-db5c-4d02-a7ff-a92cf718cb15" Jul 14 17:26:19.382306 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:19.382153 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctxxr" podUID="b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb" Jul 14 17:26:19.382306 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:19.382251 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88sg9" podUID="69371a04-a55d-4548-a7df-2868cd880309" Jul 14 17:26:19.640530 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:19.640451 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf9sf" event={"ID":"dc814136-ef4b-4cb5-a9c1-dd078e83b2f1","Type":"ContainerStarted","Data":"bfbeb245fe06d6db1ccc2fdd4a5f097bf21d728ccc2f5fe186eb9a9ae27ebaf9"} Jul 14 17:26:19.642195 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:19.642168 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-c7d6p" event={"ID":"d2bc074a-026c-402d-ac70-7d39ff8dc81f","Type":"ContainerStarted","Data":"d94935a314d1c0578254a4d29caee46b04cdc6bb8019d52148b9835c284a544f"} Jul 14 17:26:19.645293 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:19.645265 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pk9h2_f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726/ovn-acl-logging/0.log" Jul 14 17:26:19.645765 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:19.645739 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" event={"ID":"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726","Type":"ContainerStarted","Data":"0d3fa67178a0f5d471d49a6a62be295f1458b2a5797484a88a7b6ddccf1e74be"} Jul 14 17:26:19.645873 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:19.645816 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" event={"ID":"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726","Type":"ContainerStarted","Data":"66037e83eab58d6fa5603bd8aa4546a6a769f4a8a00b329827c76fea4971634f"} Jul 14 17:26:19.647806 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:19.647771 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-13.ec2.internal" event={"ID":"93bf474703caedf9ae57eedb84b07448","Type":"ContainerStarted","Data":"9d134ba0816afc949e632d5fc378ba433d82fedf896c952fd367da83159707d6"} Jul 14 17:26:19.658668 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:19.657124 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-c7d6p" podStartSLOduration=3.015253532 podStartE2EDuration="22.657105171s" totalImagesPullingTime="19.641851639s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:25:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:26:19.65691338 +0000 UTC m=+23.798162085" watchObservedRunningTime="2026-07-14 17:26:19.657105171 +0000 UTC m=+23.798353869" Jul 14 17:26:19.671968 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:19.671316 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-13.ec2.internal" podStartSLOduration=22.671296303 podStartE2EDuration="22.671296303s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:25:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:26:19.670843138 +0000 UTC m=+23.812091843" watchObservedRunningTime="2026-07-14 17:26:19.671296303 +0000 UTC m=+23.812545009" Jul 14 17:26:20.128437 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:20.128386 2554 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 14 17:26:20.280297 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:20.280151 2554 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-14T17:26:20.128404849Z","UUID":"df837087-5f29-4b83-88b0-4384127673fe","Handler":null,"Name":"","Endpoint":""} Jul 14 17:26:20.283023 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:20.283004 2554 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 14 17:26:20.283158 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:20.283057 2554 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 14 17:26:20.652748 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:20.652660 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf9sf" event={"ID":"dc814136-ef4b-4cb5-a9c1-dd078e83b2f1","Type":"ContainerStarted","Data":"1b2e32509c0b3f1dd9a5c94361786ba3195a02fc592015fff4a9a642f2fbb20b"} Jul 14 17:26:21.381906 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:21.381867 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:26:21.382066 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:21.381867 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4lhqc" Jul 14 17:26:21.382066 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:21.381973 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88sg9" podUID="69371a04-a55d-4548-a7df-2868cd880309" Jul 14 17:26:21.382177 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:21.382078 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4lhqc" podUID="56fc04ea-db5c-4d02-a7ff-a92cf718cb15" Jul 14 17:26:21.382177 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:21.381867 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctxxr" Jul 14 17:26:21.382279 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:21.382185 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctxxr" podUID="b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb" Jul 14 17:26:21.657303 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:21.657048 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf9sf" event={"ID":"dc814136-ef4b-4cb5-a9c1-dd078e83b2f1","Type":"ContainerStarted","Data":"2e9b5a5528baa9c50ed336e69d40824d9c450243b3af8f72f3b89293b592bf62"} Jul 14 17:26:21.662411 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:21.662388 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pk9h2_f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726/ovn-acl-logging/0.log" Jul 14 17:26:21.662770 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:21.662743 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" event={"ID":"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726","Type":"ContainerStarted","Data":"6d40b4f8cd69436a4a95568a44f4e238b7af60cfa4dfb18c0e87e621b0fb1e3e"} Jul 14 17:26:21.678928 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:21.678874 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf9sf" podStartSLOduration=1.182974759 podStartE2EDuration="24.678855788s" totalImagesPullingTime="23.495881029s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:25:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=3 observedRunningTime="2026-07-14 17:26:21.678242885 +0000 UTC m=+25.819491588" watchObservedRunningTime="2026-07-14 17:26:21.678855788 +0000 UTC m=+25.820104493" Jul 14 17:26:22.995223 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:22.995181 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-b8gnk_7768649b-c82e-4a04-a4c1-93061292bcec/dns-node-resolver/0.log" Jul 14 17:26:23.197071 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:23.197029 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-djms6" Jul 14 17:26:23.197761 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:23.197737 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-djms6" Jul 14 17:26:23.382188 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:23.382168 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4lhqc" Jul 14 17:26:23.382269 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:23.382180 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:26:23.382269 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:23.382168 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctxxr" Jul 14 17:26:23.382580 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:23.382291 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4lhqc" podUID="56fc04ea-db5c-4d02-a7ff-a92cf718cb15" Jul 14 17:26:23.382580 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:23.382387 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88sg9" podUID="69371a04-a55d-4548-a7df-2868cd880309" Jul 14 17:26:23.382580 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:23.382490 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctxxr" podUID="b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb" Jul 14 17:26:23.671791 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:23.671725 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pk9h2_f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726/ovn-acl-logging/0.log" Jul 14 17:26:23.672167 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:23.672134 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" event={"ID":"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726","Type":"ContainerStarted","Data":"13378c676a0dcf9099276cedb274fade04abef0f98a9a073a33047a07b521055"} Jul 14 17:26:23.672398 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:23.672378 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:26:23.672638 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:23.672613 2554 scope.go:117] "RemoveContainer" containerID="159ac3e05c8580462cdb692a64576922a5f09efab59f4b1d45ebfe65210aaeb3" Jul 14 17:26:23.674121 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:23.674101 2554 generic.go:356] "Generic (PLEG): container finished" podID="814adb62-03ce-494d-9e1f-1b78f45b87e2" containerID="cb7901673e3c8ad2b034a2098c852a965767d29d5669bd4fafcae3dee1b2418e" exitCode=0 Jul 14 17:26:23.674238 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:23.674181 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k9kwq" event={"ID":"814adb62-03ce-494d-9e1f-1b78f45b87e2","Type":"ContainerDied","Data":"cb7901673e3c8ad2b034a2098c852a965767d29d5669bd4fafcae3dee1b2418e"} Jul 14 17:26:23.690865 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:23.690658 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:26:23.783592 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:23.783569 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7ffq4_7f7817bc-7352-4f6a-9f20-cf101d130a7c/node-ca/0.log" Jul 14 17:26:24.681946 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:24.681918 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pk9h2_f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726/ovn-acl-logging/0.log" Jul 14 17:26:24.683591 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:24.683543 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" event={"ID":"f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726","Type":"ContainerStarted","Data":"07faee4cd3b7672d769174f7ab37fc5b959c7dc24860ca1e6498b38367e98c81"} Jul 14 17:26:24.683719 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:24.683661 2554 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 14 17:26:24.684028 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:24.684000 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:26:24.699279 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:24.699258 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:26:24.713239 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:24.713183 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" podStartSLOduration=7.786427092 podStartE2EDuration="27.713172154s" totalImagesPullingTime="19.926745062s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:25:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:26:24.712936611 +0000 UTC m=+28.854185314" watchObservedRunningTime="2026-07-14 17:26:24.713172154 +0000 UTC m=+28.854420857" Jul 14 17:26:25.269130 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:25.269061 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:26:25.348735 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:25.348710 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4lhqc"] Jul 14 17:26:25.348875 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:25.348807 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4lhqc" Jul 14 17:26:25.348913 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:25.348881 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4lhqc" podUID="56fc04ea-db5c-4d02-a7ff-a92cf718cb15" Jul 14 17:26:25.351381 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:25.351356 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-88sg9"] Jul 14 17:26:25.351504 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:25.351464 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:26:25.351584 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:25.351562 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88sg9" podUID="69371a04-a55d-4548-a7df-2868cd880309" Jul 14 17:26:25.352060 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:25.352011 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ctxxr"] Jul 14 17:26:25.352134 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:25.352105 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctxxr" Jul 14 17:26:25.352263 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:25.352204 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctxxr" podUID="b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb" Jul 14 17:26:25.687944 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:25.687856 2554 generic.go:356] "Generic (PLEG): container finished" podID="814adb62-03ce-494d-9e1f-1b78f45b87e2" containerID="c25bc681e660c36d819c08c511ae369b14ffebebd7a06fbd8c0c4cbf45b5a2fb" exitCode=0 Jul 14 17:26:25.688457 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:25.687939 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k9kwq" event={"ID":"814adb62-03ce-494d-9e1f-1b78f45b87e2","Type":"ContainerDied","Data":"c25bc681e660c36d819c08c511ae369b14ffebebd7a06fbd8c0c4cbf45b5a2fb"} Jul 14 17:26:26.823813 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:26.823776 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-djms6" Jul 14 17:26:26.824276 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:26.823898 2554 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 14 17:26:26.824448 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:26.824431 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-djms6" Jul 14 17:26:27.382498 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:27.382327 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:26:27.382603 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:27.382350 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctxxr" Jul 14 17:26:27.382603 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:27.382573 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88sg9" podUID="69371a04-a55d-4548-a7df-2868cd880309" Jul 14 17:26:27.382699 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:27.382378 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4lhqc" Jul 14 17:26:27.382699 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:27.382654 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctxxr" podUID="b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb" Jul 14 17:26:27.382794 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:27.382748 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4lhqc" podUID="56fc04ea-db5c-4d02-a7ff-a92cf718cb15" Jul 14 17:26:27.694350 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:27.694272 2554 generic.go:356] "Generic (PLEG): container finished" podID="814adb62-03ce-494d-9e1f-1b78f45b87e2" containerID="3337ca9645e00afce67ea932b2e4e6e3d6966b45c9efc09ed06f64d43825fdce" exitCode=0 Jul 14 17:26:27.694350 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:27.694332 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k9kwq" event={"ID":"814adb62-03ce-494d-9e1f-1b78f45b87e2","Type":"ContainerDied","Data":"3337ca9645e00afce67ea932b2e4e6e3d6966b45c9efc09ed06f64d43825fdce"} Jul 14 17:26:29.100831 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:29.100797 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69371a04-a55d-4548-a7df-2868cd880309-original-pull-secret\") pod \"global-pull-secret-syncer-88sg9\" (UID: \"69371a04-a55d-4548-a7df-2868cd880309\") " pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:26:29.101560 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:29.100849 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb-metrics-certs\") pod \"network-metrics-daemon-ctxxr\" (UID: \"b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb\") " pod="openshift-multus/network-metrics-daemon-ctxxr" Jul 14 17:26:29.101560 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:29.100961 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:26:29.101560 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:29.100975 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 17:26:29.101560 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:29.101023 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb-metrics-certs podName:b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb nodeName:}" failed. No retries permitted until 2026-07-14 17:27:01.101005365 +0000 UTC m=+65.242254064 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb-metrics-certs") pod "network-metrics-daemon-ctxxr" (UID: "b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:26:29.101560 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:29.101041 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69371a04-a55d-4548-a7df-2868cd880309-original-pull-secret podName:69371a04-a55d-4548-a7df-2868cd880309 nodeName:}" failed. No retries permitted until 2026-07-14 17:27:01.101032505 +0000 UTC m=+65.242281192 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/69371a04-a55d-4548-a7df-2868cd880309-original-pull-secret") pod "global-pull-secret-syncer-88sg9" (UID: "69371a04-a55d-4548-a7df-2868cd880309") : object "kube-system"/"original-pull-secret" not registered Jul 14 17:26:29.201730 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:29.201699 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq99q\" (UniqueName: \"kubernetes.io/projected/56fc04ea-db5c-4d02-a7ff-a92cf718cb15-kube-api-access-nq99q\") pod \"network-check-target-4lhqc\" (UID: \"56fc04ea-db5c-4d02-a7ff-a92cf718cb15\") " pod="openshift-network-diagnostics/network-check-target-4lhqc" Jul 14 17:26:29.201898 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:29.201841 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 17:26:29.201898 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:29.201863 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 17:26:29.201898 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:29.201875 2554 projected.go:196] Error preparing data for projected volume kube-api-access-nq99q for pod openshift-network-diagnostics/network-check-target-4lhqc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:26:29.202110 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:29.201932 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/56fc04ea-db5c-4d02-a7ff-a92cf718cb15-kube-api-access-nq99q podName:56fc04ea-db5c-4d02-a7ff-a92cf718cb15 nodeName:}" failed. No retries permitted until 2026-07-14 17:27:01.201914907 +0000 UTC m=+65.343163610 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-nq99q" (UniqueName: "kubernetes.io/projected/56fc04ea-db5c-4d02-a7ff-a92cf718cb15-kube-api-access-nq99q") pod "network-check-target-4lhqc" (UID: "56fc04ea-db5c-4d02-a7ff-a92cf718cb15") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:26:29.382681 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:29.382600 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctxxr" Jul 14 17:26:29.382681 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:29.382625 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:26:29.382862 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:29.382603 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4lhqc" Jul 14 17:26:29.382862 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:29.382734 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctxxr" podUID="b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb" Jul 14 17:26:29.382862 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:29.382810 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4lhqc" podUID="56fc04ea-db5c-4d02-a7ff-a92cf718cb15" Jul 14 17:26:29.383028 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:29.382911 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88sg9" podUID="69371a04-a55d-4548-a7df-2868cd880309" Jul 14 17:26:31.382591 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:31.382539 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4lhqc" Jul 14 17:26:31.383204 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:31.382556 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:26:31.383204 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:31.382690 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctxxr" Jul 14 17:26:31.383204 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:31.382693 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4lhqc" podUID="56fc04ea-db5c-4d02-a7ff-a92cf718cb15" Jul 14 17:26:31.383204 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:31.382792 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88sg9" podUID="69371a04-a55d-4548-a7df-2868cd880309" Jul 14 17:26:31.383204 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:31.382878 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctxxr" podUID="b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb" Jul 14 17:26:32.709131 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:32.709098 2554 generic.go:356] "Generic (PLEG): container finished" podID="814adb62-03ce-494d-9e1f-1b78f45b87e2" containerID="c16ccb0d530ed6b47ccb0b82ae52310eb114b081d0d70c5738c85dfe95cf8181" exitCode=0 Jul 14 17:26:32.709517 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:32.709158 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k9kwq" event={"ID":"814adb62-03ce-494d-9e1f-1b78f45b87e2","Type":"ContainerDied","Data":"c16ccb0d530ed6b47ccb0b82ae52310eb114b081d0d70c5738c85dfe95cf8181"} Jul 14 17:26:33.381944 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:33.381914 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4lhqc" Jul 14 17:26:33.381944 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:33.381946 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctxxr" Jul 14 17:26:33.382131 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:33.381950 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:26:33.382131 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:33.382022 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4lhqc" podUID="56fc04ea-db5c-4d02-a7ff-a92cf718cb15" Jul 14 17:26:33.382131 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:33.382095 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctxxr" podUID="b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb" Jul 14 17:26:33.382239 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:33.382190 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88sg9" podUID="69371a04-a55d-4548-a7df-2868cd880309" Jul 14 17:26:33.714284 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:33.714189 2554 generic.go:356] "Generic (PLEG): container finished" podID="814adb62-03ce-494d-9e1f-1b78f45b87e2" containerID="6fc7fa12090928ab497df3fd2b8d84f19be927bcf584be14982b0ebc9e6e356a" exitCode=0 Jul 14 17:26:33.714284 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:33.714261 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k9kwq" event={"ID":"814adb62-03ce-494d-9e1f-1b78f45b87e2","Type":"ContainerDied","Data":"6fc7fa12090928ab497df3fd2b8d84f19be927bcf584be14982b0ebc9e6e356a"} Jul 14 17:26:34.719576 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:34.719541 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k9kwq" event={"ID":"814adb62-03ce-494d-9e1f-1b78f45b87e2","Type":"ContainerStarted","Data":"4b120ebcf4514c8127098138d59c9d2830f718d8b4d34db7a86906af0741eac3"} Jul 14 17:26:35.382397 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:35.382172 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4lhqc" Jul 14 17:26:35.382541 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:35.382226 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:26:35.382541 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:35.382471 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4lhqc" podUID="56fc04ea-db5c-4d02-a7ff-a92cf718cb15" Jul 14 17:26:35.382647 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:35.382238 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctxxr" Jul 14 17:26:35.382647 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:35.382548 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88sg9" podUID="69371a04-a55d-4548-a7df-2868cd880309" Jul 14 17:26:35.382647 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:35.382627 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctxxr" podUID="b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb" Jul 14 17:26:37.382548 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:37.382516 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:26:37.382940 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:37.382618 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctxxr" Jul 14 17:26:37.382940 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:37.382629 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88sg9" podUID="69371a04-a55d-4548-a7df-2868cd880309" Jul 14 17:26:37.382940 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:37.382644 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4lhqc" Jul 14 17:26:37.382940 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:37.382723 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctxxr" podUID="b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb" Jul 14 17:26:37.382940 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:37.382785 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4lhqc" podUID="56fc04ea-db5c-4d02-a7ff-a92cf718cb15" Jul 14 17:26:39.381709 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:39.381680 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:26:39.382055 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:39.381768 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctxxr" Jul 14 17:26:39.382055 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:39.381772 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88sg9" podUID="69371a04-a55d-4548-a7df-2868cd880309" Jul 14 17:26:39.382055 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:39.381777 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4lhqc" Jul 14 17:26:39.382055 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:39.381869 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctxxr" podUID="b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb" Jul 14 17:26:39.382055 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:39.381925 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4lhqc" podUID="56fc04ea-db5c-4d02-a7ff-a92cf718cb15" Jul 14 17:26:41.381943 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:41.381915 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4lhqc" Jul 14 17:26:41.381943 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:41.381950 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctxxr" Jul 14 17:26:41.382483 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:41.381953 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:26:41.382483 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:41.382031 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4lhqc" podUID="56fc04ea-db5c-4d02-a7ff-a92cf718cb15" Jul 14 17:26:41.382483 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:41.382137 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctxxr" podUID="b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb" Jul 14 17:26:41.382483 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:41.382207 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88sg9" podUID="69371a04-a55d-4548-a7df-2868cd880309" Jul 14 17:26:43.382233 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:43.382180 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctxxr" Jul 14 17:26:43.382233 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:43.382204 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4lhqc" Jul 14 17:26:43.382747 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:43.382180 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:26:43.382747 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:43.382323 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctxxr" podUID="b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb" Jul 14 17:26:43.382747 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:43.382396 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4lhqc" podUID="56fc04ea-db5c-4d02-a7ff-a92cf718cb15" Jul 14 17:26:43.382747 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:43.382484 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88sg9" podUID="69371a04-a55d-4548-a7df-2868cd880309" Jul 14 17:26:45.140826 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.140799 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-13.ec2.internal" event="NodeReady" Jul 14 17:26:45.141303 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.140976 2554 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 14 17:26:45.183035 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.182940 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-k9kwq" podStartSLOduration=17.175857889 podStartE2EDuration="48.182923475s" totalImagesPullingTime="31.007065586s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:25:57 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:26:34.746788164 +0000 UTC m=+38.888036867" watchObservedRunningTime="2026-07-14 17:26:45.182923475 +0000 UTC m=+49.324172178" Jul 14 17:26:45.183528 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.183512 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-hbrtv"] Jul 14 17:26:45.198838 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.198812 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-bwrcq"] Jul 14 17:26:45.198969 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.198945 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hbrtv" Jul 14 17:26:45.201701 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.201679 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 14 17:26:45.201823 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.201686 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6ft2z\"" Jul 14 17:26:45.202067 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.202054 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 14 17:26:45.212763 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.212740 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hbrtv"] Jul 14 17:26:45.212889 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.212776 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lzjdm"] Jul 14 17:26:45.212889 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.212878 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bwrcq" Jul 14 17:26:45.215725 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.215706 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 14 17:26:45.215834 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.215816 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 14 17:26:45.215834 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.215821 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 14 17:26:45.215961 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.215933 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2lrmp\"" Jul 14 17:26:45.222351 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.222327 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb86b8d6-ce99-4d8a-8170-2c421c00208a-tmp-dir\") pod \"dns-default-hbrtv\" (UID: \"cb86b8d6-ce99-4d8a-8170-2c421c00208a\") " pod="openshift-dns/dns-default-hbrtv" Jul 14 17:26:45.222459 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.222358 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpjmv\" (UniqueName: \"kubernetes.io/projected/cb86b8d6-ce99-4d8a-8170-2c421c00208a-kube-api-access-jpjmv\") pod \"dns-default-hbrtv\" (UID: \"cb86b8d6-ce99-4d8a-8170-2c421c00208a\") " pod="openshift-dns/dns-default-hbrtv" Jul 14 17:26:45.222670 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.222649 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cb86b8d6-ce99-4d8a-8170-2c421c00208a-config-volume\") pod \"dns-default-hbrtv\" (UID: \"cb86b8d6-ce99-4d8a-8170-2c421c00208a\") " pod="openshift-dns/dns-default-hbrtv" Jul 14 17:26:45.222761 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.222689 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb86b8d6-ce99-4d8a-8170-2c421c00208a-metrics-tls\") pod \"dns-default-hbrtv\" (UID: \"cb86b8d6-ce99-4d8a-8170-2c421c00208a\") " pod="openshift-dns/dns-default-hbrtv" Jul 14 17:26:45.230867 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.230848 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bwrcq"] Jul 14 17:26:45.230948 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.230881 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lzjdm"] Jul 14 17:26:45.230994 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.230967 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lzjdm" Jul 14 17:26:45.234015 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.233994 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 14 17:26:45.234207 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.234187 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 14 17:26:45.234427 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.234073 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 14 17:26:45.234680 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.234664 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 14 17:26:45.234889 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.234724 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nk7rg\"" Jul 14 17:26:45.323133 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.323097 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e360557-5281-4fd8-b27a-7ec139ccc92b-cert\") pod \"ingress-canary-bwrcq\" (UID: \"8e360557-5281-4fd8-b27a-7ec139ccc92b\") " pod="openshift-ingress-canary/ingress-canary-bwrcq" Jul 14 17:26:45.323133 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.323137 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b038901e-a28d-4de8-a035-194c93348967-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lzjdm\" (UID: \"b038901e-a28d-4de8-a035-194c93348967\") " pod="openshift-insights/insights-runtime-extractor-lzjdm" Jul 14 17:26:45.323385 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.323163 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lm9cz\" (UniqueName: \"kubernetes.io/projected/b038901e-a28d-4de8-a035-194c93348967-kube-api-access-lm9cz\") pod \"insights-runtime-extractor-lzjdm\" (UID: \"b038901e-a28d-4de8-a035-194c93348967\") " pod="openshift-insights/insights-runtime-extractor-lzjdm" Jul 14 17:26:45.323385 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.323258 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cb86b8d6-ce99-4d8a-8170-2c421c00208a-config-volume\") pod \"dns-default-hbrtv\" (UID: \"cb86b8d6-ce99-4d8a-8170-2c421c00208a\") " pod="openshift-dns/dns-default-hbrtv" Jul 14 17:26:45.323385 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.323290 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb86b8d6-ce99-4d8a-8170-2c421c00208a-metrics-tls\") pod \"dns-default-hbrtv\" (UID: \"cb86b8d6-ce99-4d8a-8170-2c421c00208a\") " pod="openshift-dns/dns-default-hbrtv" Jul 14 17:26:45.323385 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.323316 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qs4zr\" (UniqueName: \"kubernetes.io/projected/8e360557-5281-4fd8-b27a-7ec139ccc92b-kube-api-access-qs4zr\") pod \"ingress-canary-bwrcq\" (UID: \"8e360557-5281-4fd8-b27a-7ec139ccc92b\") " pod="openshift-ingress-canary/ingress-canary-bwrcq" Jul 14 17:26:45.323385 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.323349 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b038901e-a28d-4de8-a035-194c93348967-data-volume\") pod \"insights-runtime-extractor-lzjdm\" (UID: \"b038901e-a28d-4de8-a035-194c93348967\") " pod="openshift-insights/insights-runtime-extractor-lzjdm" Jul 14 17:26:45.323385 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.323374 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b038901e-a28d-4de8-a035-194c93348967-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lzjdm\" (UID: \"b038901e-a28d-4de8-a035-194c93348967\") " pod="openshift-insights/insights-runtime-extractor-lzjdm" Jul 14 17:26:45.323670 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.323397 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb86b8d6-ce99-4d8a-8170-2c421c00208a-tmp-dir\") pod \"dns-default-hbrtv\" (UID: \"cb86b8d6-ce99-4d8a-8170-2c421c00208a\") " pod="openshift-dns/dns-default-hbrtv" Jul 14 17:26:45.323670 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.323411 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpjmv\" (UniqueName: \"kubernetes.io/projected/cb86b8d6-ce99-4d8a-8170-2c421c00208a-kube-api-access-jpjmv\") pod \"dns-default-hbrtv\" (UID: \"cb86b8d6-ce99-4d8a-8170-2c421c00208a\") " pod="openshift-dns/dns-default-hbrtv" Jul 14 17:26:45.323670 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.323450 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b038901e-a28d-4de8-a035-194c93348967-crio-socket\") pod \"insights-runtime-extractor-lzjdm\" (UID: \"b038901e-a28d-4de8-a035-194c93348967\") " pod="openshift-insights/insights-runtime-extractor-lzjdm" Jul 14 17:26:45.323821 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.323721 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb86b8d6-ce99-4d8a-8170-2c421c00208a-tmp-dir\") pod \"dns-default-hbrtv\" (UID: \"cb86b8d6-ce99-4d8a-8170-2c421c00208a\") " pod="openshift-dns/dns-default-hbrtv" Jul 14 17:26:45.323987 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.323967 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cb86b8d6-ce99-4d8a-8170-2c421c00208a-config-volume\") pod \"dns-default-hbrtv\" (UID: \"cb86b8d6-ce99-4d8a-8170-2c421c00208a\") " pod="openshift-dns/dns-default-hbrtv" Jul 14 17:26:45.327350 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.327332 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb86b8d6-ce99-4d8a-8170-2c421c00208a-metrics-tls\") pod \"dns-default-hbrtv\" (UID: \"cb86b8d6-ce99-4d8a-8170-2c421c00208a\") " pod="openshift-dns/dns-default-hbrtv" Jul 14 17:26:45.331315 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.331292 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpjmv\" (UniqueName: \"kubernetes.io/projected/cb86b8d6-ce99-4d8a-8170-2c421c00208a-kube-api-access-jpjmv\") pod \"dns-default-hbrtv\" (UID: \"cb86b8d6-ce99-4d8a-8170-2c421c00208a\") " pod="openshift-dns/dns-default-hbrtv" Jul 14 17:26:45.382356 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.382306 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:26:45.382356 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.382362 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4lhqc" Jul 14 17:26:45.382608 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.382460 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctxxr" Jul 14 17:26:45.386194 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.386167 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 14 17:26:45.386194 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.386178 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 14 17:26:45.386398 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.386167 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 14 17:26:45.386398 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.386202 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ph225\"" Jul 14 17:26:45.386398 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.386178 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hwsmr\"" Jul 14 17:26:45.386398 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.386178 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 14 17:26:45.424292 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.424261 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b038901e-a28d-4de8-a035-194c93348967-data-volume\") pod \"insights-runtime-extractor-lzjdm\" (UID: \"b038901e-a28d-4de8-a035-194c93348967\") " pod="openshift-insights/insights-runtime-extractor-lzjdm" Jul 14 17:26:45.424292 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.424293 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b038901e-a28d-4de8-a035-194c93348967-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lzjdm\" (UID: \"b038901e-a28d-4de8-a035-194c93348967\") " pod="openshift-insights/insights-runtime-extractor-lzjdm" Jul 14 17:26:45.424489 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.424318 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b038901e-a28d-4de8-a035-194c93348967-crio-socket\") pod \"insights-runtime-extractor-lzjdm\" (UID: \"b038901e-a28d-4de8-a035-194c93348967\") " pod="openshift-insights/insights-runtime-extractor-lzjdm" Jul 14 17:26:45.424489 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.424363 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e360557-5281-4fd8-b27a-7ec139ccc92b-cert\") pod \"ingress-canary-bwrcq\" (UID: \"8e360557-5281-4fd8-b27a-7ec139ccc92b\") " pod="openshift-ingress-canary/ingress-canary-bwrcq" Jul 14 17:26:45.424489 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.424379 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b038901e-a28d-4de8-a035-194c93348967-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lzjdm\" (UID: \"b038901e-a28d-4de8-a035-194c93348967\") " pod="openshift-insights/insights-runtime-extractor-lzjdm" Jul 14 17:26:45.424489 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.424397 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lm9cz\" (UniqueName: \"kubernetes.io/projected/b038901e-a28d-4de8-a035-194c93348967-kube-api-access-lm9cz\") pod \"insights-runtime-extractor-lzjdm\" (UID: \"b038901e-a28d-4de8-a035-194c93348967\") " pod="openshift-insights/insights-runtime-extractor-lzjdm" Jul 14 17:26:45.424489 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.424424 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qs4zr\" (UniqueName: \"kubernetes.io/projected/8e360557-5281-4fd8-b27a-7ec139ccc92b-kube-api-access-qs4zr\") pod \"ingress-canary-bwrcq\" (UID: \"8e360557-5281-4fd8-b27a-7ec139ccc92b\") " pod="openshift-ingress-canary/ingress-canary-bwrcq" Jul 14 17:26:45.424674 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.424583 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b038901e-a28d-4de8-a035-194c93348967-crio-socket\") pod \"insights-runtime-extractor-lzjdm\" (UID: \"b038901e-a28d-4de8-a035-194c93348967\") " pod="openshift-insights/insights-runtime-extractor-lzjdm" Jul 14 17:26:45.424720 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.424699 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b038901e-a28d-4de8-a035-194c93348967-data-volume\") pod \"insights-runtime-extractor-lzjdm\" (UID: \"b038901e-a28d-4de8-a035-194c93348967\") " pod="openshift-insights/insights-runtime-extractor-lzjdm" Jul 14 17:26:45.424904 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.424884 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b038901e-a28d-4de8-a035-194c93348967-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lzjdm\" (UID: \"b038901e-a28d-4de8-a035-194c93348967\") " pod="openshift-insights/insights-runtime-extractor-lzjdm" Jul 14 17:26:45.426684 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.426666 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b038901e-a28d-4de8-a035-194c93348967-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lzjdm\" (UID: \"b038901e-a28d-4de8-a035-194c93348967\") " pod="openshift-insights/insights-runtime-extractor-lzjdm" Jul 14 17:26:45.426784 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.426700 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e360557-5281-4fd8-b27a-7ec139ccc92b-cert\") pod \"ingress-canary-bwrcq\" (UID: \"8e360557-5281-4fd8-b27a-7ec139ccc92b\") " pod="openshift-ingress-canary/ingress-canary-bwrcq" Jul 14 17:26:45.431814 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.431792 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lm9cz\" (UniqueName: \"kubernetes.io/projected/b038901e-a28d-4de8-a035-194c93348967-kube-api-access-lm9cz\") pod \"insights-runtime-extractor-lzjdm\" (UID: \"b038901e-a28d-4de8-a035-194c93348967\") " pod="openshift-insights/insights-runtime-extractor-lzjdm" Jul 14 17:26:45.431924 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.431883 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qs4zr\" (UniqueName: \"kubernetes.io/projected/8e360557-5281-4fd8-b27a-7ec139ccc92b-kube-api-access-qs4zr\") pod \"ingress-canary-bwrcq\" (UID: \"8e360557-5281-4fd8-b27a-7ec139ccc92b\") " pod="openshift-ingress-canary/ingress-canary-bwrcq" Jul 14 17:26:45.509046 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.508965 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hbrtv" Jul 14 17:26:45.521123 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.521102 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bwrcq" Jul 14 17:26:45.540424 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.540401 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lzjdm" Jul 14 17:26:45.732033 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.731994 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hbrtv"] Jul 14 17:26:45.736439 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:26:45.736402 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb86b8d6_ce99_4d8a_8170_2c421c00208a.slice/crio-93b9200f3bfbb711e1398b9a1a9c0c02aa27593e0c580e81b1757ff66f0109af WatchSource:0}: Error finding container 93b9200f3bfbb711e1398b9a1a9c0c02aa27593e0c580e81b1757ff66f0109af: Status 404 returned error can't find the container with id 93b9200f3bfbb711e1398b9a1a9c0c02aa27593e0c580e81b1757ff66f0109af Jul 14 17:26:45.744406 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.744374 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hbrtv" event={"ID":"cb86b8d6-ce99-4d8a-8170-2c421c00208a","Type":"ContainerStarted","Data":"93b9200f3bfbb711e1398b9a1a9c0c02aa27593e0c580e81b1757ff66f0109af"} Jul 14 17:26:45.747760 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.747735 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bwrcq"] Jul 14 17:26:45.750744 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:26:45.750720 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e360557_5281_4fd8_b27a_7ec139ccc92b.slice/crio-0e9001100b61853f6f38c9c3229900a94b439f74817ae848c6262218b3dc957c WatchSource:0}: Error finding container 0e9001100b61853f6f38c9c3229900a94b439f74817ae848c6262218b3dc957c: Status 404 returned error can't find the container with id 0e9001100b61853f6f38c9c3229900a94b439f74817ae848c6262218b3dc957c Jul 14 17:26:45.766650 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:45.766573 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lzjdm"] Jul 14 17:26:45.770086 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:26:45.770063 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb038901e_a28d_4de8_a035_194c93348967.slice/crio-39b6070eee39907ffaa9231ef3506127ee026de8aa2139373a86acb917d80e7b WatchSource:0}: Error finding container 39b6070eee39907ffaa9231ef3506127ee026de8aa2139373a86acb917d80e7b: Status 404 returned error can't find the container with id 39b6070eee39907ffaa9231ef3506127ee026de8aa2139373a86acb917d80e7b Jul 14 17:26:46.752298 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:46.752176 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lzjdm" event={"ID":"b038901e-a28d-4de8-a035-194c93348967","Type":"ContainerStarted","Data":"39b6070eee39907ffaa9231ef3506127ee026de8aa2139373a86acb917d80e7b"} Jul 14 17:26:46.754547 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:46.754517 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bwrcq" event={"ID":"8e360557-5281-4fd8-b27a-7ec139ccc92b","Type":"ContainerStarted","Data":"0e9001100b61853f6f38c9c3229900a94b439f74817ae848c6262218b3dc957c"} Jul 14 17:26:52.770921 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:52.770882 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lzjdm" event={"ID":"b038901e-a28d-4de8-a035-194c93348967","Type":"ContainerStarted","Data":"b51e63bb8c00dcb56bca4034224ce590f7409eb3e2cae29f9ae2e6b34094244c"} Jul 14 17:26:52.772252 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:52.772204 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bwrcq" event={"ID":"8e360557-5281-4fd8-b27a-7ec139ccc92b","Type":"ContainerStarted","Data":"525fab26edbb2c24a64002f035ba6950b25e62705e29839de5fbaa4ec66e86be"} Jul 14 17:26:52.776988 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:52.776960 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hbrtv" event={"ID":"cb86b8d6-ce99-4d8a-8170-2c421c00208a","Type":"ContainerStarted","Data":"c5c8f6cd3383124db7a7b2675246ed0c7a5fd32fc1a52ba794534fe3e43f2d1e"} Jul 14 17:26:52.777078 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:52.777039 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-hbrtv" Jul 14 17:26:52.777078 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:52.777057 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hbrtv" event={"ID":"cb86b8d6-ce99-4d8a-8170-2c421c00208a","Type":"ContainerStarted","Data":"75f576805d0f373d450a3bd30219b9cc0b74530eabd16438a8262dc6fde323b8"} Jul 14 17:26:52.787078 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:52.787030 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-bwrcq" podStartSLOduration=1.722906587 podStartE2EDuration="7.787015649s" totalImagesPullingTime="6.064109062s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:26:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:26:52.786618677 +0000 UTC m=+56.927867381" watchObservedRunningTime="2026-07-14 17:26:52.787015649 +0000 UTC m=+56.928264354" Jul 14 17:26:52.801414 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:52.801368 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-hbrtv" podStartSLOduration=1.7225758020000002 podStartE2EDuration="7.801353142s" totalImagesPullingTime="6.07877734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:26:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:26:52.80120726 +0000 UTC m=+56.942455965" watchObservedRunningTime="2026-07-14 17:26:52.801353142 +0000 UTC m=+56.942601845" Jul 14 17:26:53.782278 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:53.782247 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lzjdm" event={"ID":"b038901e-a28d-4de8-a035-194c93348967","Type":"ContainerStarted","Data":"ec467bbddce13239c289b8b390014ae306e6806bc1de972f41faa5a4ec34cab9"} Jul 14 17:26:56.700390 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:56.700359 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pk9h2" Jul 14 17:26:56.793393 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:56.793355 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lzjdm" event={"ID":"b038901e-a28d-4de8-a035-194c93348967","Type":"ContainerStarted","Data":"73a01823e705b999294c54e1a260c6cf9244c6db2a433d5df0f1a4e09535c174"} Jul 14 17:26:56.816241 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:56.815939 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lzjdm" podStartSLOduration=1.786209159 podStartE2EDuration="11.815919709s" totalImagesPullingTime="10.02971055s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:26:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:26:56.813310737 +0000 UTC m=+60.954559454" watchObservedRunningTime="2026-07-14 17:26:56.815919709 +0000 UTC m=+60.957168414" Jul 14 17:26:57.114558 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:57.114526 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-hlw49"] Jul 14 17:26:57.116537 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:57.116521 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-hlw49" Jul 14 17:26:57.118780 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:57.118762 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-tqnnv\"" Jul 14 17:26:57.118889 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:57.118875 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 14 17:26:57.124119 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:57.124095 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-hlw49"] Jul 14 17:26:57.211580 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:57.211547 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1dd534d0-8b90-42c4-8d00-8e11079fb474-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-hlw49\" (UID: \"1dd534d0-8b90-42c4-8d00-8e11079fb474\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-hlw49" Jul 14 17:26:57.312166 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:57.312131 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1dd534d0-8b90-42c4-8d00-8e11079fb474-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-hlw49\" (UID: \"1dd534d0-8b90-42c4-8d00-8e11079fb474\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-hlw49" Jul 14 17:26:57.312311 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:57.312293 2554 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 14 17:26:57.312375 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:26:57.312365 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dd534d0-8b90-42c4-8d00-8e11079fb474-tls-certificates podName:1dd534d0-8b90-42c4-8d00-8e11079fb474 nodeName:}" failed. No retries permitted until 2026-07-14 17:26:57.812350365 +0000 UTC m=+61.953599047 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/1dd534d0-8b90-42c4-8d00-8e11079fb474-tls-certificates") pod "prometheus-operator-admission-webhook-77d98bc786-hlw49" (UID: "1dd534d0-8b90-42c4-8d00-8e11079fb474") : secret "prometheus-operator-admission-webhook-tls" not found Jul 14 17:26:57.815109 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:57.815083 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1dd534d0-8b90-42c4-8d00-8e11079fb474-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-hlw49\" (UID: \"1dd534d0-8b90-42c4-8d00-8e11079fb474\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-hlw49" Jul 14 17:26:57.817463 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:57.817444 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1dd534d0-8b90-42c4-8d00-8e11079fb474-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-hlw49\" (UID: \"1dd534d0-8b90-42c4-8d00-8e11079fb474\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-hlw49" Jul 14 17:26:58.025230 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:58.025182 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-hlw49" Jul 14 17:26:58.029641 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:58.029614 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-vgj5f"] Jul 14 17:26:58.031692 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:58.031677 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-vgj5f" Jul 14 17:26:58.034177 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:58.034155 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-8vnr8\"" Jul 14 17:26:58.034304 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:58.034185 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 14 17:26:58.034304 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:58.034187 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 14 17:26:58.040909 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:58.040888 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-vgj5f"] Jul 14 17:26:58.116731 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:58.116703 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2ms5\" (UniqueName: \"kubernetes.io/projected/9415d694-c5e5-492d-a2f2-348323c40cd7-kube-api-access-n2ms5\") pod \"downloads-68c9cbb497-vgj5f\" (UID: \"9415d694-c5e5-492d-a2f2-348323c40cd7\") " pod="openshift-console/downloads-68c9cbb497-vgj5f" Jul 14 17:26:58.148958 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:58.148934 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-hlw49"] Jul 14 17:26:58.152423 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:26:58.152399 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dd534d0_8b90_42c4_8d00_8e11079fb474.slice/crio-0ea443f4c48857f498d6142d0417bf9997e4b7f70713ea24fed3dd54fe453c20 WatchSource:0}: Error finding container 0ea443f4c48857f498d6142d0417bf9997e4b7f70713ea24fed3dd54fe453c20: Status 404 returned error can't find the container with id 0ea443f4c48857f498d6142d0417bf9997e4b7f70713ea24fed3dd54fe453c20 Jul 14 17:26:58.217181 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:58.217156 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2ms5\" (UniqueName: \"kubernetes.io/projected/9415d694-c5e5-492d-a2f2-348323c40cd7-kube-api-access-n2ms5\") pod \"downloads-68c9cbb497-vgj5f\" (UID: \"9415d694-c5e5-492d-a2f2-348323c40cd7\") " pod="openshift-console/downloads-68c9cbb497-vgj5f" Jul 14 17:26:58.224686 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:58.224665 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2ms5\" (UniqueName: \"kubernetes.io/projected/9415d694-c5e5-492d-a2f2-348323c40cd7-kube-api-access-n2ms5\") pod \"downloads-68c9cbb497-vgj5f\" (UID: \"9415d694-c5e5-492d-a2f2-348323c40cd7\") " pod="openshift-console/downloads-68c9cbb497-vgj5f" Jul 14 17:26:58.349258 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:58.349139 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-vgj5f" Jul 14 17:26:58.464489 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:58.464468 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-vgj5f"] Jul 14 17:26:58.466683 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:26:58.466656 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9415d694_c5e5_492d_a2f2_348323c40cd7.slice/crio-a0b19440813b92063c6948c224090b195bba883fb539f79e533f85380d97b500 WatchSource:0}: Error finding container a0b19440813b92063c6948c224090b195bba883fb539f79e533f85380d97b500: Status 404 returned error can't find the container with id a0b19440813b92063c6948c224090b195bba883fb539f79e533f85380d97b500 Jul 14 17:26:58.800533 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:58.800495 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-vgj5f" event={"ID":"9415d694-c5e5-492d-a2f2-348323c40cd7","Type":"ContainerStarted","Data":"a0b19440813b92063c6948c224090b195bba883fb539f79e533f85380d97b500"} Jul 14 17:26:58.801634 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:58.801606 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-hlw49" event={"ID":"1dd534d0-8b90-42c4-8d00-8e11079fb474","Type":"ContainerStarted","Data":"0ea443f4c48857f498d6142d0417bf9997e4b7f70713ea24fed3dd54fe453c20"} Jul 14 17:26:59.805837 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:59.805751 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-hlw49" event={"ID":"1dd534d0-8b90-42c4-8d00-8e11079fb474","Type":"ContainerStarted","Data":"aa9ba9a86fe7cec6d75490d3535b3f3356afa45822e1b482020587be59550aba"} Jul 14 17:26:59.806233 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:59.805878 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-hlw49" Jul 14 17:26:59.810537 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:59.810517 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-hlw49" Jul 14 17:26:59.824912 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:26:59.824871 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-hlw49" podStartSLOduration=1.424401267 podStartE2EDuration="2.824859499s" totalImagesPullingTime="1.400458232s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:26:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:26:59.824338374 +0000 UTC m=+63.965587089" watchObservedRunningTime="2026-07-14 17:26:59.824859499 +0000 UTC m=+63.966108205" Jul 14 17:27:00.174079 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:00.174044 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-8hl55"] Jul 14 17:27:00.176150 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:00.176135 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-8hl55" Jul 14 17:27:00.179556 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:00.179533 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 14 17:27:00.179688 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:00.179537 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 14 17:27:00.179688 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:00.179637 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 14 17:27:00.179940 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:00.179726 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 14 17:27:00.180303 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:00.180284 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-8bk5n\"" Jul 14 17:27:00.180422 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:00.180286 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 14 17:27:00.186115 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:00.186088 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-8hl55"] Jul 14 17:27:00.231095 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:00.231062 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxz5c\" (UniqueName: \"kubernetes.io/projected/da755c27-b962-4855-8616-aa05a3bdab0f-kube-api-access-cxz5c\") pod \"prometheus-operator-64785b6675-8hl55\" (UID: \"da755c27-b962-4855-8616-aa05a3bdab0f\") " pod="openshift-monitoring/prometheus-operator-64785b6675-8hl55" Jul 14 17:27:00.231252 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:00.231107 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/da755c27-b962-4855-8616-aa05a3bdab0f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-8hl55\" (UID: \"da755c27-b962-4855-8616-aa05a3bdab0f\") " pod="openshift-monitoring/prometheus-operator-64785b6675-8hl55" Jul 14 17:27:00.231252 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:00.231233 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/da755c27-b962-4855-8616-aa05a3bdab0f-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-8hl55\" (UID: \"da755c27-b962-4855-8616-aa05a3bdab0f\") " pod="openshift-monitoring/prometheus-operator-64785b6675-8hl55" Jul 14 17:27:00.231364 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:00.231262 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da755c27-b962-4855-8616-aa05a3bdab0f-metrics-client-ca\") pod \"prometheus-operator-64785b6675-8hl55\" (UID: \"da755c27-b962-4855-8616-aa05a3bdab0f\") " pod="openshift-monitoring/prometheus-operator-64785b6675-8hl55" Jul 14 17:27:00.331937 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:00.331902 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/da755c27-b962-4855-8616-aa05a3bdab0f-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-8hl55\" (UID: \"da755c27-b962-4855-8616-aa05a3bdab0f\") " pod="openshift-monitoring/prometheus-operator-64785b6675-8hl55" Jul 14 17:27:00.331937 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:00.331938 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da755c27-b962-4855-8616-aa05a3bdab0f-metrics-client-ca\") pod \"prometheus-operator-64785b6675-8hl55\" (UID: \"da755c27-b962-4855-8616-aa05a3bdab0f\") " pod="openshift-monitoring/prometheus-operator-64785b6675-8hl55" Jul 14 17:27:00.332130 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:00.331995 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxz5c\" (UniqueName: \"kubernetes.io/projected/da755c27-b962-4855-8616-aa05a3bdab0f-kube-api-access-cxz5c\") pod \"prometheus-operator-64785b6675-8hl55\" (UID: \"da755c27-b962-4855-8616-aa05a3bdab0f\") " pod="openshift-monitoring/prometheus-operator-64785b6675-8hl55" Jul 14 17:27:00.332130 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:00.332016 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/da755c27-b962-4855-8616-aa05a3bdab0f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-8hl55\" (UID: \"da755c27-b962-4855-8616-aa05a3bdab0f\") " pod="openshift-monitoring/prometheus-operator-64785b6675-8hl55" Jul 14 17:27:00.332653 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:00.332631 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da755c27-b962-4855-8616-aa05a3bdab0f-metrics-client-ca\") pod \"prometheus-operator-64785b6675-8hl55\" (UID: \"da755c27-b962-4855-8616-aa05a3bdab0f\") " pod="openshift-monitoring/prometheus-operator-64785b6675-8hl55" Jul 14 17:27:00.334365 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:00.334340 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/da755c27-b962-4855-8616-aa05a3bdab0f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-8hl55\" (UID: \"da755c27-b962-4855-8616-aa05a3bdab0f\") " pod="openshift-monitoring/prometheus-operator-64785b6675-8hl55" Jul 14 17:27:00.334451 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:00.334401 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/da755c27-b962-4855-8616-aa05a3bdab0f-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-8hl55\" (UID: \"da755c27-b962-4855-8616-aa05a3bdab0f\") " pod="openshift-monitoring/prometheus-operator-64785b6675-8hl55" Jul 14 17:27:00.339462 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:00.339444 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxz5c\" (UniqueName: \"kubernetes.io/projected/da755c27-b962-4855-8616-aa05a3bdab0f-kube-api-access-cxz5c\") pod \"prometheus-operator-64785b6675-8hl55\" (UID: \"da755c27-b962-4855-8616-aa05a3bdab0f\") " pod="openshift-monitoring/prometheus-operator-64785b6675-8hl55" Jul 14 17:27:00.485469 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:00.485396 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-8hl55" Jul 14 17:27:00.613249 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:00.613201 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-8hl55"] Jul 14 17:27:00.617159 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:27:00.617132 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda755c27_b962_4855_8616_aa05a3bdab0f.slice/crio-4a0a70565a3fd855bf0358a4b3bd7d48580858f0d8bbf752a5d3377b5e51c8fa WatchSource:0}: Error finding container 4a0a70565a3fd855bf0358a4b3bd7d48580858f0d8bbf752a5d3377b5e51c8fa: Status 404 returned error can't find the container with id 4a0a70565a3fd855bf0358a4b3bd7d48580858f0d8bbf752a5d3377b5e51c8fa Jul 14 17:27:00.811034 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:00.810949 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-8hl55" event={"ID":"da755c27-b962-4855-8616-aa05a3bdab0f","Type":"ContainerStarted","Data":"4a0a70565a3fd855bf0358a4b3bd7d48580858f0d8bbf752a5d3377b5e51c8fa"} Jul 14 17:27:01.138516 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:01.138477 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69371a04-a55d-4548-a7df-2868cd880309-original-pull-secret\") pod \"global-pull-secret-syncer-88sg9\" (UID: \"69371a04-a55d-4548-a7df-2868cd880309\") " pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:27:01.138676 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:01.138537 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb-metrics-certs\") pod \"network-metrics-daemon-ctxxr\" (UID: \"b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb\") " pod="openshift-multus/network-metrics-daemon-ctxxr" Jul 14 17:27:01.141289 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:01.141263 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 14 17:27:01.141389 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:01.141295 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 14 17:27:01.151986 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:01.151963 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69371a04-a55d-4548-a7df-2868cd880309-original-pull-secret\") pod \"global-pull-secret-syncer-88sg9\" (UID: \"69371a04-a55d-4548-a7df-2868cd880309\") " pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:27:01.152098 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:01.152067 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb-metrics-certs\") pod \"network-metrics-daemon-ctxxr\" (UID: \"b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb\") " pod="openshift-multus/network-metrics-daemon-ctxxr" Jul 14 17:27:01.239281 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:01.239247 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq99q\" (UniqueName: \"kubernetes.io/projected/56fc04ea-db5c-4d02-a7ff-a92cf718cb15-kube-api-access-nq99q\") pod \"network-check-target-4lhqc\" (UID: \"56fc04ea-db5c-4d02-a7ff-a92cf718cb15\") " pod="openshift-network-diagnostics/network-check-target-4lhqc" Jul 14 17:27:01.241827 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:01.241806 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 14 17:27:01.252061 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:01.252044 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 14 17:27:01.263139 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:01.263111 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nq99q\" (UniqueName: \"kubernetes.io/projected/56fc04ea-db5c-4d02-a7ff-a92cf718cb15-kube-api-access-nq99q\") pod \"network-check-target-4lhqc\" (UID: \"56fc04ea-db5c-4d02-a7ff-a92cf718cb15\") " pod="openshift-network-diagnostics/network-check-target-4lhqc" Jul 14 17:27:01.291641 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:01.291623 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88sg9" Jul 14 17:27:01.300126 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:01.300110 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hwsmr\"" Jul 14 17:27:01.304925 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:01.304905 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ph225\"" Jul 14 17:27:01.308643 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:01.308626 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctxxr" Jul 14 17:27:01.313771 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:01.313756 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4lhqc" Jul 14 17:27:01.445720 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:01.445281 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-88sg9"] Jul 14 17:27:01.451849 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:27:01.450450 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69371a04_a55d_4548_a7df_2868cd880309.slice/crio-fd81fd5547909af36cf2276a0573019e3caaef8b7ee96fc1a6af45f700c4aa1e WatchSource:0}: Error finding container fd81fd5547909af36cf2276a0573019e3caaef8b7ee96fc1a6af45f700c4aa1e: Status 404 returned error can't find the container with id fd81fd5547909af36cf2276a0573019e3caaef8b7ee96fc1a6af45f700c4aa1e Jul 14 17:27:01.475174 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:01.475132 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ctxxr"] Jul 14 17:27:01.478968 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:27:01.478941 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7fc6cf1_c9dc_426b_865d_cd12a16c4dcb.slice/crio-678fa48e92039f4cf78f3858cee75110604c728994623456ef062c15e8dea33b WatchSource:0}: Error finding container 678fa48e92039f4cf78f3858cee75110604c728994623456ef062c15e8dea33b: Status 404 returned error can't find the container with id 678fa48e92039f4cf78f3858cee75110604c728994623456ef062c15e8dea33b Jul 14 17:27:01.492856 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:01.492828 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4lhqc"] Jul 14 17:27:01.495870 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:27:01.495836 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56fc04ea_db5c_4d02_a7ff_a92cf718cb15.slice/crio-4b545a4d02c2a90e508fbe2c2c4ce76881415e90f67caaaf253d56f08ee727d5 WatchSource:0}: Error finding container 4b545a4d02c2a90e508fbe2c2c4ce76881415e90f67caaaf253d56f08ee727d5: Status 404 returned error can't find the container with id 4b545a4d02c2a90e508fbe2c2c4ce76881415e90f67caaaf253d56f08ee727d5 Jul 14 17:27:01.816588 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:01.816503 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4lhqc" event={"ID":"56fc04ea-db5c-4d02-a7ff-a92cf718cb15","Type":"ContainerStarted","Data":"4b545a4d02c2a90e508fbe2c2c4ce76881415e90f67caaaf253d56f08ee727d5"} Jul 14 17:27:01.817852 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:01.817823 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-88sg9" event={"ID":"69371a04-a55d-4548-a7df-2868cd880309","Type":"ContainerStarted","Data":"fd81fd5547909af36cf2276a0573019e3caaef8b7ee96fc1a6af45f700c4aa1e"} Jul 14 17:27:01.819264 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:01.819237 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ctxxr" event={"ID":"b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb","Type":"ContainerStarted","Data":"678fa48e92039f4cf78f3858cee75110604c728994623456ef062c15e8dea33b"} Jul 14 17:27:02.784941 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:02.784903 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-hbrtv" Jul 14 17:27:02.844687 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:02.844460 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-8hl55" event={"ID":"da755c27-b962-4855-8616-aa05a3bdab0f","Type":"ContainerStarted","Data":"bf64f20f7631fb04ea2219915f3f7bf036eed0953426ab686b059b49ab8ebb7d"} Jul 14 17:27:02.849893 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:02.849813 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ctxxr" event={"ID":"b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb","Type":"ContainerStarted","Data":"1c00b0c0394addd5913a0f73d6d013f39a01c69caddfaa8fc38e9db796bf08e0"} Jul 14 17:27:03.860044 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:03.860004 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-8hl55" event={"ID":"da755c27-b962-4855-8616-aa05a3bdab0f","Type":"ContainerStarted","Data":"01104777054ed8dbddf7769ebef48056107cc974bb33105adf563ee46f5e8821"} Jul 14 17:27:03.864545 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:03.864513 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ctxxr" event={"ID":"b7fc6cf1-c9dc-426b-865d-cd12a16c4dcb","Type":"ContainerStarted","Data":"3dffac55e8d6407d74828fc020fd2bec38809176c9fffa1757216d99df94a793"} Jul 14 17:27:03.878545 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:03.877263 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-8hl55" podStartSLOduration=1.841022479 podStartE2EDuration="3.877246985s" totalImagesPullingTime="2.036224506s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:27:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:27:03.876748732 +0000 UTC m=+68.017997437" watchObservedRunningTime="2026-07-14 17:27:03.877246985 +0000 UTC m=+68.018495690" Jul 14 17:27:03.893049 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:03.892735 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ctxxr" podStartSLOduration=65.716218312 podStartE2EDuration="1m6.892715809s" totalImagesPullingTime="1.176497497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:25:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:27:03.890666376 +0000 UTC m=+68.031915087" watchObservedRunningTime="2026-07-14 17:27:03.892715809 +0000 UTC m=+68.033964517" Jul 14 17:27:05.553774 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:05.553711 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-lglc8"] Jul 14 17:27:05.582526 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:05.582497 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lglc8" Jul 14 17:27:05.585470 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:05.585442 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 14 17:27:05.585470 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:05.585459 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 14 17:27:05.585647 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:05.585464 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zfrgx\"" Jul 14 17:27:05.585647 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:05.585442 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 14 17:27:05.669808 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:05.669776 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/798ec85b-c0a7-4738-a711-d72c0c6c8b4a-node-exporter-accelerators-collector-config\") pod \"node-exporter-lglc8\" (UID: \"798ec85b-c0a7-4738-a711-d72c0c6c8b4a\") " pod="openshift-monitoring/node-exporter-lglc8" Jul 14 17:27:05.669973 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:05.669817 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zttvm\" (UniqueName: \"kubernetes.io/projected/798ec85b-c0a7-4738-a711-d72c0c6c8b4a-kube-api-access-zttvm\") pod \"node-exporter-lglc8\" (UID: \"798ec85b-c0a7-4738-a711-d72c0c6c8b4a\") " pod="openshift-monitoring/node-exporter-lglc8" Jul 14 17:27:05.669973 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:05.669854 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/798ec85b-c0a7-4738-a711-d72c0c6c8b4a-node-exporter-tls\") pod \"node-exporter-lglc8\" (UID: \"798ec85b-c0a7-4738-a711-d72c0c6c8b4a\") " pod="openshift-monitoring/node-exporter-lglc8" Jul 14 17:27:05.669973 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:05.669909 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/798ec85b-c0a7-4738-a711-d72c0c6c8b4a-node-exporter-wtmp\") pod \"node-exporter-lglc8\" (UID: \"798ec85b-c0a7-4738-a711-d72c0c6c8b4a\") " pod="openshift-monitoring/node-exporter-lglc8" Jul 14 17:27:05.669973 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:05.669949 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/798ec85b-c0a7-4738-a711-d72c0c6c8b4a-sys\") pod \"node-exporter-lglc8\" (UID: \"798ec85b-c0a7-4738-a711-d72c0c6c8b4a\") " pod="openshift-monitoring/node-exporter-lglc8" Jul 14 17:27:05.670182 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:05.669970 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/798ec85b-c0a7-4738-a711-d72c0c6c8b4a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lglc8\" (UID: \"798ec85b-c0a7-4738-a711-d72c0c6c8b4a\") " pod="openshift-monitoring/node-exporter-lglc8" Jul 14 17:27:05.670182 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:05.670003 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/798ec85b-c0a7-4738-a711-d72c0c6c8b4a-root\") pod \"node-exporter-lglc8\" (UID: \"798ec85b-c0a7-4738-a711-d72c0c6c8b4a\") " pod="openshift-monitoring/node-exporter-lglc8" Jul 14 17:27:05.670182 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:05.670027 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/798ec85b-c0a7-4738-a711-d72c0c6c8b4a-node-exporter-textfile\") pod \"node-exporter-lglc8\" (UID: \"798ec85b-c0a7-4738-a711-d72c0c6c8b4a\") " pod="openshift-monitoring/node-exporter-lglc8" Jul 14 17:27:05.670182 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:05.670050 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/798ec85b-c0a7-4738-a711-d72c0c6c8b4a-metrics-client-ca\") pod \"node-exporter-lglc8\" (UID: \"798ec85b-c0a7-4738-a711-d72c0c6c8b4a\") " pod="openshift-monitoring/node-exporter-lglc8" Jul 14 17:27:05.770920 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:05.770888 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/798ec85b-c0a7-4738-a711-d72c0c6c8b4a-node-exporter-accelerators-collector-config\") pod \"node-exporter-lglc8\" (UID: \"798ec85b-c0a7-4738-a711-d72c0c6c8b4a\") " pod="openshift-monitoring/node-exporter-lglc8" Jul 14 17:27:05.770920 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:05.770921 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zttvm\" (UniqueName: \"kubernetes.io/projected/798ec85b-c0a7-4738-a711-d72c0c6c8b4a-kube-api-access-zttvm\") pod \"node-exporter-lglc8\" (UID: \"798ec85b-c0a7-4738-a711-d72c0c6c8b4a\") " pod="openshift-monitoring/node-exporter-lglc8" Jul 14 17:27:05.771165 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:05.770949 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/798ec85b-c0a7-4738-a711-d72c0c6c8b4a-node-exporter-tls\") pod \"node-exporter-lglc8\" (UID: \"798ec85b-c0a7-4738-a711-d72c0c6c8b4a\") " pod="openshift-monitoring/node-exporter-lglc8" Jul 14 17:27:05.771165 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:05.770982 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/798ec85b-c0a7-4738-a711-d72c0c6c8b4a-node-exporter-wtmp\") pod \"node-exporter-lglc8\" (UID: \"798ec85b-c0a7-4738-a711-d72c0c6c8b4a\") " pod="openshift-monitoring/node-exporter-lglc8" Jul 14 17:27:05.771165 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:05.771111 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/798ec85b-c0a7-4738-a711-d72c0c6c8b4a-node-exporter-wtmp\") pod \"node-exporter-lglc8\" (UID: \"798ec85b-c0a7-4738-a711-d72c0c6c8b4a\") " pod="openshift-monitoring/node-exporter-lglc8" Jul 14 17:27:05.771353 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:05.771298 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/798ec85b-c0a7-4738-a711-d72c0c6c8b4a-sys\") pod \"node-exporter-lglc8\" (UID: \"798ec85b-c0a7-4738-a711-d72c0c6c8b4a\") " pod="openshift-monitoring/node-exporter-lglc8" Jul 14 17:27:05.771353 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:05.771332 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/798ec85b-c0a7-4738-a711-d72c0c6c8b4a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lglc8\" (UID: \"798ec85b-c0a7-4738-a711-d72c0c6c8b4a\") " pod="openshift-monitoring/node-exporter-lglc8" Jul 14 17:27:05.771448 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:05.771369 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/798ec85b-c0a7-4738-a711-d72c0c6c8b4a-root\") pod \"node-exporter-lglc8\" (UID: \"798ec85b-c0a7-4738-a711-d72c0c6c8b4a\") " pod="openshift-monitoring/node-exporter-lglc8" Jul 14 17:27:05.771448 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:05.771392 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/798ec85b-c0a7-4738-a711-d72c0c6c8b4a-node-exporter-textfile\") pod \"node-exporter-lglc8\" (UID: \"798ec85b-c0a7-4738-a711-d72c0c6c8b4a\") " pod="openshift-monitoring/node-exporter-lglc8" Jul 14 17:27:05.771448 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:05.771417 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/798ec85b-c0a7-4738-a711-d72c0c6c8b4a-metrics-client-ca\") pod \"node-exporter-lglc8\" (UID: \"798ec85b-c0a7-4738-a711-d72c0c6c8b4a\") " pod="openshift-monitoring/node-exporter-lglc8" Jul 14 17:27:05.771590 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:05.771570 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/798ec85b-c0a7-4738-a711-d72c0c6c8b4a-sys\") pod \"node-exporter-lglc8\" (UID: \"798ec85b-c0a7-4738-a711-d72c0c6c8b4a\") " pod="openshift-monitoring/node-exporter-lglc8" Jul 14 17:27:05.771644 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:05.771622 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/798ec85b-c0a7-4738-a711-d72c0c6c8b4a-root\") pod \"node-exporter-lglc8\" (UID: \"798ec85b-c0a7-4738-a711-d72c0c6c8b4a\") " pod="openshift-monitoring/node-exporter-lglc8" Jul 14 17:27:05.771736 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:05.771703 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/798ec85b-c0a7-4738-a711-d72c0c6c8b4a-node-exporter-textfile\") pod \"node-exporter-lglc8\" (UID: \"798ec85b-c0a7-4738-a711-d72c0c6c8b4a\") " pod="openshift-monitoring/node-exporter-lglc8" Jul 14 17:27:05.771860 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:05.771737 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/798ec85b-c0a7-4738-a711-d72c0c6c8b4a-node-exporter-accelerators-collector-config\") pod \"node-exporter-lglc8\" (UID: \"798ec85b-c0a7-4738-a711-d72c0c6c8b4a\") " pod="openshift-monitoring/node-exporter-lglc8" Jul 14 17:27:05.772059 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:05.771995 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/798ec85b-c0a7-4738-a711-d72c0c6c8b4a-metrics-client-ca\") pod \"node-exporter-lglc8\" (UID: \"798ec85b-c0a7-4738-a711-d72c0c6c8b4a\") " pod="openshift-monitoring/node-exporter-lglc8" Jul 14 17:27:05.773946 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:05.773920 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/798ec85b-c0a7-4738-a711-d72c0c6c8b4a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lglc8\" (UID: \"798ec85b-c0a7-4738-a711-d72c0c6c8b4a\") " pod="openshift-monitoring/node-exporter-lglc8" Jul 14 17:27:05.774020 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:05.773925 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/798ec85b-c0a7-4738-a711-d72c0c6c8b4a-node-exporter-tls\") pod \"node-exporter-lglc8\" (UID: \"798ec85b-c0a7-4738-a711-d72c0c6c8b4a\") " pod="openshift-monitoring/node-exporter-lglc8" Jul 14 17:27:05.782495 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:05.782469 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zttvm\" (UniqueName: \"kubernetes.io/projected/798ec85b-c0a7-4738-a711-d72c0c6c8b4a-kube-api-access-zttvm\") pod \"node-exporter-lglc8\" (UID: \"798ec85b-c0a7-4738-a711-d72c0c6c8b4a\") " pod="openshift-monitoring/node-exporter-lglc8" Jul 14 17:27:05.891881 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:05.891855 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lglc8" Jul 14 17:27:07.340326 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:27:07.340281 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod798ec85b_c0a7_4738_a711_d72c0c6c8b4a.slice/crio-6b6b04aab354be656ee0856176f4f8f659939f5c42a481eb999dedf78ebf510c WatchSource:0}: Error finding container 6b6b04aab354be656ee0856176f4f8f659939f5c42a481eb999dedf78ebf510c: Status 404 returned error can't find the container with id 6b6b04aab354be656ee0856176f4f8f659939f5c42a481eb999dedf78ebf510c Jul 14 17:27:07.507769 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.507726 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-777fb8f587-jwff6"] Jul 14 17:27:07.510612 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.510583 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-777fb8f587-jwff6" Jul 14 17:27:07.513315 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.513149 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 14 17:27:07.513315 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.513186 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-z4ft8\"" Jul 14 17:27:07.513315 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.513206 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 14 17:27:07.513503 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.513491 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-3m3imiu8iiu3o\"" Jul 14 17:27:07.513737 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.513561 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 14 17:27:07.513737 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.513664 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 14 17:27:07.514241 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.514155 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 14 17:27:07.521672 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.521595 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-777fb8f587-jwff6"] Jul 14 17:27:07.584381 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.584350 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ab2cd57e-c525-4003-99b4-488ca8f02e9e-secret-thanos-querier-tls\") pod \"thanos-querier-777fb8f587-jwff6\" (UID: \"ab2cd57e-c525-4003-99b4-488ca8f02e9e\") " pod="openshift-monitoring/thanos-querier-777fb8f587-jwff6" Jul 14 17:27:07.584567 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.584415 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ab2cd57e-c525-4003-99b4-488ca8f02e9e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-777fb8f587-jwff6\" (UID: \"ab2cd57e-c525-4003-99b4-488ca8f02e9e\") " pod="openshift-monitoring/thanos-querier-777fb8f587-jwff6" Jul 14 17:27:07.584567 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.584457 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ab2cd57e-c525-4003-99b4-488ca8f02e9e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-777fb8f587-jwff6\" (UID: \"ab2cd57e-c525-4003-99b4-488ca8f02e9e\") " pod="openshift-monitoring/thanos-querier-777fb8f587-jwff6" Jul 14 17:27:07.584567 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.584518 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ab2cd57e-c525-4003-99b4-488ca8f02e9e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-777fb8f587-jwff6\" (UID: \"ab2cd57e-c525-4003-99b4-488ca8f02e9e\") " pod="openshift-monitoring/thanos-querier-777fb8f587-jwff6" Jul 14 17:27:07.584567 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.584560 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab2cd57e-c525-4003-99b4-488ca8f02e9e-metrics-client-ca\") pod \"thanos-querier-777fb8f587-jwff6\" (UID: \"ab2cd57e-c525-4003-99b4-488ca8f02e9e\") " pod="openshift-monitoring/thanos-querier-777fb8f587-jwff6" Jul 14 17:27:07.584760 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.584587 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndg6d\" (UniqueName: \"kubernetes.io/projected/ab2cd57e-c525-4003-99b4-488ca8f02e9e-kube-api-access-ndg6d\") pod \"thanos-querier-777fb8f587-jwff6\" (UID: \"ab2cd57e-c525-4003-99b4-488ca8f02e9e\") " pod="openshift-monitoring/thanos-querier-777fb8f587-jwff6" Jul 14 17:27:07.584760 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.584654 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ab2cd57e-c525-4003-99b4-488ca8f02e9e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-777fb8f587-jwff6\" (UID: \"ab2cd57e-c525-4003-99b4-488ca8f02e9e\") " pod="openshift-monitoring/thanos-querier-777fb8f587-jwff6" Jul 14 17:27:07.584760 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.584678 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ab2cd57e-c525-4003-99b4-488ca8f02e9e-secret-grpc-tls\") pod \"thanos-querier-777fb8f587-jwff6\" (UID: \"ab2cd57e-c525-4003-99b4-488ca8f02e9e\") " pod="openshift-monitoring/thanos-querier-777fb8f587-jwff6" Jul 14 17:27:07.685895 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.685855 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ab2cd57e-c525-4003-99b4-488ca8f02e9e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-777fb8f587-jwff6\" (UID: \"ab2cd57e-c525-4003-99b4-488ca8f02e9e\") " pod="openshift-monitoring/thanos-querier-777fb8f587-jwff6" Jul 14 17:27:07.686200 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.685914 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ab2cd57e-c525-4003-99b4-488ca8f02e9e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-777fb8f587-jwff6\" (UID: \"ab2cd57e-c525-4003-99b4-488ca8f02e9e\") " pod="openshift-monitoring/thanos-querier-777fb8f587-jwff6" Jul 14 17:27:07.686200 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.685951 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ab2cd57e-c525-4003-99b4-488ca8f02e9e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-777fb8f587-jwff6\" (UID: \"ab2cd57e-c525-4003-99b4-488ca8f02e9e\") " pod="openshift-monitoring/thanos-querier-777fb8f587-jwff6" Jul 14 17:27:07.686200 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.685978 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab2cd57e-c525-4003-99b4-488ca8f02e9e-metrics-client-ca\") pod \"thanos-querier-777fb8f587-jwff6\" (UID: \"ab2cd57e-c525-4003-99b4-488ca8f02e9e\") " pod="openshift-monitoring/thanos-querier-777fb8f587-jwff6" Jul 14 17:27:07.686200 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.686003 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndg6d\" (UniqueName: \"kubernetes.io/projected/ab2cd57e-c525-4003-99b4-488ca8f02e9e-kube-api-access-ndg6d\") pod \"thanos-querier-777fb8f587-jwff6\" (UID: \"ab2cd57e-c525-4003-99b4-488ca8f02e9e\") " pod="openshift-monitoring/thanos-querier-777fb8f587-jwff6" Jul 14 17:27:07.686200 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.686042 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ab2cd57e-c525-4003-99b4-488ca8f02e9e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-777fb8f587-jwff6\" (UID: \"ab2cd57e-c525-4003-99b4-488ca8f02e9e\") " pod="openshift-monitoring/thanos-querier-777fb8f587-jwff6" Jul 14 17:27:07.686200 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.686067 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ab2cd57e-c525-4003-99b4-488ca8f02e9e-secret-grpc-tls\") pod \"thanos-querier-777fb8f587-jwff6\" (UID: \"ab2cd57e-c525-4003-99b4-488ca8f02e9e\") " pod="openshift-monitoring/thanos-querier-777fb8f587-jwff6" Jul 14 17:27:07.686200 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.686108 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ab2cd57e-c525-4003-99b4-488ca8f02e9e-secret-thanos-querier-tls\") pod \"thanos-querier-777fb8f587-jwff6\" (UID: \"ab2cd57e-c525-4003-99b4-488ca8f02e9e\") " pod="openshift-monitoring/thanos-querier-777fb8f587-jwff6" Jul 14 17:27:07.686841 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.686788 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab2cd57e-c525-4003-99b4-488ca8f02e9e-metrics-client-ca\") pod \"thanos-querier-777fb8f587-jwff6\" (UID: \"ab2cd57e-c525-4003-99b4-488ca8f02e9e\") " pod="openshift-monitoring/thanos-querier-777fb8f587-jwff6" Jul 14 17:27:07.689197 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.689156 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ab2cd57e-c525-4003-99b4-488ca8f02e9e-secret-thanos-querier-tls\") pod \"thanos-querier-777fb8f587-jwff6\" (UID: \"ab2cd57e-c525-4003-99b4-488ca8f02e9e\") " pod="openshift-monitoring/thanos-querier-777fb8f587-jwff6" Jul 14 17:27:07.689197 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.689156 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ab2cd57e-c525-4003-99b4-488ca8f02e9e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-777fb8f587-jwff6\" (UID: \"ab2cd57e-c525-4003-99b4-488ca8f02e9e\") " pod="openshift-monitoring/thanos-querier-777fb8f587-jwff6" Jul 14 17:27:07.689392 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.689329 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ab2cd57e-c525-4003-99b4-488ca8f02e9e-secret-grpc-tls\") pod \"thanos-querier-777fb8f587-jwff6\" (UID: \"ab2cd57e-c525-4003-99b4-488ca8f02e9e\") " pod="openshift-monitoring/thanos-querier-777fb8f587-jwff6" Jul 14 17:27:07.689894 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.689875 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ab2cd57e-c525-4003-99b4-488ca8f02e9e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-777fb8f587-jwff6\" (UID: \"ab2cd57e-c525-4003-99b4-488ca8f02e9e\") " pod="openshift-monitoring/thanos-querier-777fb8f587-jwff6" Jul 14 17:27:07.689992 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.689927 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ab2cd57e-c525-4003-99b4-488ca8f02e9e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-777fb8f587-jwff6\" (UID: \"ab2cd57e-c525-4003-99b4-488ca8f02e9e\") " pod="openshift-monitoring/thanos-querier-777fb8f587-jwff6" Jul 14 17:27:07.691073 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.691050 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ab2cd57e-c525-4003-99b4-488ca8f02e9e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-777fb8f587-jwff6\" (UID: \"ab2cd57e-c525-4003-99b4-488ca8f02e9e\") " pod="openshift-monitoring/thanos-querier-777fb8f587-jwff6" Jul 14 17:27:07.693892 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.693868 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndg6d\" (UniqueName: \"kubernetes.io/projected/ab2cd57e-c525-4003-99b4-488ca8f02e9e-kube-api-access-ndg6d\") pod \"thanos-querier-777fb8f587-jwff6\" (UID: \"ab2cd57e-c525-4003-99b4-488ca8f02e9e\") " pod="openshift-monitoring/thanos-querier-777fb8f587-jwff6" Jul 14 17:27:07.824573 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.824539 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-777fb8f587-jwff6" Jul 14 17:27:07.886347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.885927 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4lhqc" event={"ID":"56fc04ea-db5c-4d02-a7ff-a92cf718cb15","Type":"ContainerStarted","Data":"ac8b293a7536709b8bf176229a3ccdac9d87b8ae39d340a7352b687fe18888c8"} Jul 14 17:27:07.886347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.886304 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-4lhqc" Jul 14 17:27:07.888134 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.888086 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-88sg9" event={"ID":"69371a04-a55d-4548-a7df-2868cd880309","Type":"ContainerStarted","Data":"2c914854acffd46fa52e5afdd4f5b1e8eaf5c0ea21e2588e63ad285bc49e40ba"} Jul 14 17:27:07.890335 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.890166 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lglc8" event={"ID":"798ec85b-c0a7-4738-a711-d72c0c6c8b4a","Type":"ContainerStarted","Data":"6b6b04aab354be656ee0856176f4f8f659939f5c42a481eb999dedf78ebf510c"} Jul 14 17:27:07.904563 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.904469 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-4lhqc" podStartSLOduration=65.059632583 podStartE2EDuration="1m10.904449662s" totalImagesPullingTime="5.844817079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:25:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:27:07.904050128 +0000 UTC m=+72.045298834" watchObservedRunningTime="2026-07-14 17:27:07.904449662 +0000 UTC m=+72.045698366" Jul 14 17:27:07.921762 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.921535 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-88sg9" podStartSLOduration=65.019904103 podStartE2EDuration="1m10.921515953s" totalImagesPullingTime="5.90161185s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:25:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:27:07.921300634 +0000 UTC m=+72.062549338" watchObservedRunningTime="2026-07-14 17:27:07.921515953 +0000 UTC m=+72.062764658" Jul 14 17:27:07.975250 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:07.975199 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-777fb8f587-jwff6"] Jul 14 17:27:07.977513 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:27:07.977482 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab2cd57e_c525_4003_99b4_488ca8f02e9e.slice/crio-f22f201cb6e74eade084bf5362344eb7596331a0e05562ad875f883dd77011c9 WatchSource:0}: Error finding container f22f201cb6e74eade084bf5362344eb7596331a0e05562ad875f883dd77011c9: Status 404 returned error can't find the container with id f22f201cb6e74eade084bf5362344eb7596331a0e05562ad875f883dd77011c9 Jul 14 17:27:08.896297 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:08.896256 2554 generic.go:356] "Generic (PLEG): container finished" podID="798ec85b-c0a7-4738-a711-d72c0c6c8b4a" containerID="9b30d5edeaea52428c50c2c5a33dec1cdf71d0798937f77099bf221ca5964bc0" exitCode=0 Jul 14 17:27:08.896756 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:08.896350 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lglc8" event={"ID":"798ec85b-c0a7-4738-a711-d72c0c6c8b4a","Type":"ContainerDied","Data":"9b30d5edeaea52428c50c2c5a33dec1cdf71d0798937f77099bf221ca5964bc0"} Jul 14 17:27:08.897733 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:08.897705 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-777fb8f587-jwff6" event={"ID":"ab2cd57e-c525-4003-99b4-488ca8f02e9e","Type":"ContainerStarted","Data":"f22f201cb6e74eade084bf5362344eb7596331a0e05562ad875f883dd77011c9"} Jul 14 17:27:08.956638 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:08.956605 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c66bc44b6-9d8hc"] Jul 14 17:27:08.962060 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:08.962036 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c66bc44b6-9d8hc" Jul 14 17:27:08.965450 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:08.965015 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 14 17:27:08.965450 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:08.965083 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 14 17:27:08.965450 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:08.965147 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 14 17:27:08.965450 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:08.965331 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 14 17:27:08.965450 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:08.965366 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-zlkg8\"" Jul 14 17:27:08.965450 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:08.965394 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 14 17:27:08.968887 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:08.968864 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c66bc44b6-9d8hc"] Jul 14 17:27:09.097261 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:09.097157 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff9d44ef-36ce-4659-89e3-5021b6fd3ce4-console-serving-cert\") pod \"console-7c66bc44b6-9d8hc\" (UID: \"ff9d44ef-36ce-4659-89e3-5021b6fd3ce4\") " pod="openshift-console/console-7c66bc44b6-9d8hc" Jul 14 17:27:09.097261 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:09.097201 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ff9d44ef-36ce-4659-89e3-5021b6fd3ce4-console-config\") pod \"console-7c66bc44b6-9d8hc\" (UID: \"ff9d44ef-36ce-4659-89e3-5021b6fd3ce4\") " pod="openshift-console/console-7c66bc44b6-9d8hc" Jul 14 17:27:09.097470 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:09.097288 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ff9d44ef-36ce-4659-89e3-5021b6fd3ce4-console-oauth-config\") pod \"console-7c66bc44b6-9d8hc\" (UID: \"ff9d44ef-36ce-4659-89e3-5021b6fd3ce4\") " pod="openshift-console/console-7c66bc44b6-9d8hc" Jul 14 17:27:09.097470 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:09.097356 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ff9d44ef-36ce-4659-89e3-5021b6fd3ce4-service-ca\") pod \"console-7c66bc44b6-9d8hc\" (UID: \"ff9d44ef-36ce-4659-89e3-5021b6fd3ce4\") " pod="openshift-console/console-7c66bc44b6-9d8hc" Jul 14 17:27:09.097470 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:09.097388 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sv4ph\" (UniqueName: \"kubernetes.io/projected/ff9d44ef-36ce-4659-89e3-5021b6fd3ce4-kube-api-access-sv4ph\") pod \"console-7c66bc44b6-9d8hc\" (UID: \"ff9d44ef-36ce-4659-89e3-5021b6fd3ce4\") " pod="openshift-console/console-7c66bc44b6-9d8hc" Jul 14 17:27:09.097470 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:09.097419 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ff9d44ef-36ce-4659-89e3-5021b6fd3ce4-oauth-serving-cert\") pod \"console-7c66bc44b6-9d8hc\" (UID: \"ff9d44ef-36ce-4659-89e3-5021b6fd3ce4\") " pod="openshift-console/console-7c66bc44b6-9d8hc" Jul 14 17:27:09.198635 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:09.198511 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ff9d44ef-36ce-4659-89e3-5021b6fd3ce4-oauth-serving-cert\") pod \"console-7c66bc44b6-9d8hc\" (UID: \"ff9d44ef-36ce-4659-89e3-5021b6fd3ce4\") " pod="openshift-console/console-7c66bc44b6-9d8hc" Jul 14 17:27:09.198784 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:09.198690 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff9d44ef-36ce-4659-89e3-5021b6fd3ce4-console-serving-cert\") pod \"console-7c66bc44b6-9d8hc\" (UID: \"ff9d44ef-36ce-4659-89e3-5021b6fd3ce4\") " pod="openshift-console/console-7c66bc44b6-9d8hc" Jul 14 17:27:09.198784 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:09.198742 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ff9d44ef-36ce-4659-89e3-5021b6fd3ce4-console-config\") pod \"console-7c66bc44b6-9d8hc\" (UID: \"ff9d44ef-36ce-4659-89e3-5021b6fd3ce4\") " pod="openshift-console/console-7c66bc44b6-9d8hc" Jul 14 17:27:09.198883 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:09.198774 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ff9d44ef-36ce-4659-89e3-5021b6fd3ce4-console-oauth-config\") pod \"console-7c66bc44b6-9d8hc\" (UID: \"ff9d44ef-36ce-4659-89e3-5021b6fd3ce4\") " pod="openshift-console/console-7c66bc44b6-9d8hc" Jul 14 17:27:09.198883 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:09.198828 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ff9d44ef-36ce-4659-89e3-5021b6fd3ce4-service-ca\") pod \"console-7c66bc44b6-9d8hc\" (UID: \"ff9d44ef-36ce-4659-89e3-5021b6fd3ce4\") " pod="openshift-console/console-7c66bc44b6-9d8hc" Jul 14 17:27:09.198883 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:09.198858 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv4ph\" (UniqueName: \"kubernetes.io/projected/ff9d44ef-36ce-4659-89e3-5021b6fd3ce4-kube-api-access-sv4ph\") pod \"console-7c66bc44b6-9d8hc\" (UID: \"ff9d44ef-36ce-4659-89e3-5021b6fd3ce4\") " pod="openshift-console/console-7c66bc44b6-9d8hc" Jul 14 17:27:09.199750 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:09.199718 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ff9d44ef-36ce-4659-89e3-5021b6fd3ce4-oauth-serving-cert\") pod \"console-7c66bc44b6-9d8hc\" (UID: \"ff9d44ef-36ce-4659-89e3-5021b6fd3ce4\") " pod="openshift-console/console-7c66bc44b6-9d8hc" Jul 14 17:27:09.200204 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:09.200162 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ff9d44ef-36ce-4659-89e3-5021b6fd3ce4-console-config\") pod \"console-7c66bc44b6-9d8hc\" (UID: \"ff9d44ef-36ce-4659-89e3-5021b6fd3ce4\") " pod="openshift-console/console-7c66bc44b6-9d8hc" Jul 14 17:27:09.200204 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:09.200184 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ff9d44ef-36ce-4659-89e3-5021b6fd3ce4-service-ca\") pod \"console-7c66bc44b6-9d8hc\" (UID: \"ff9d44ef-36ce-4659-89e3-5021b6fd3ce4\") " pod="openshift-console/console-7c66bc44b6-9d8hc" Jul 14 17:27:09.202107 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:09.202086 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff9d44ef-36ce-4659-89e3-5021b6fd3ce4-console-serving-cert\") pod \"console-7c66bc44b6-9d8hc\" (UID: \"ff9d44ef-36ce-4659-89e3-5021b6fd3ce4\") " pod="openshift-console/console-7c66bc44b6-9d8hc" Jul 14 17:27:09.202195 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:09.202093 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ff9d44ef-36ce-4659-89e3-5021b6fd3ce4-console-oauth-config\") pod \"console-7c66bc44b6-9d8hc\" (UID: \"ff9d44ef-36ce-4659-89e3-5021b6fd3ce4\") " pod="openshift-console/console-7c66bc44b6-9d8hc" Jul 14 17:27:09.208613 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:09.208591 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sv4ph\" (UniqueName: \"kubernetes.io/projected/ff9d44ef-36ce-4659-89e3-5021b6fd3ce4-kube-api-access-sv4ph\") pod \"console-7c66bc44b6-9d8hc\" (UID: \"ff9d44ef-36ce-4659-89e3-5021b6fd3ce4\") " pod="openshift-console/console-7c66bc44b6-9d8hc" Jul 14 17:27:09.274196 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:09.274153 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c66bc44b6-9d8hc" Jul 14 17:27:09.432551 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:09.432512 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c66bc44b6-9d8hc"] Jul 14 17:27:09.437510 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:27:09.437480 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff9d44ef_36ce_4659_89e3_5021b6fd3ce4.slice/crio-14742c3e7cecbe68a69975ba75d4b954d61a6864e48a27f4e34d8035c8beee2c WatchSource:0}: Error finding container 14742c3e7cecbe68a69975ba75d4b954d61a6864e48a27f4e34d8035c8beee2c: Status 404 returned error can't find the container with id 14742c3e7cecbe68a69975ba75d4b954d61a6864e48a27f4e34d8035c8beee2c Jul 14 17:27:09.904452 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:09.904409 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lglc8" event={"ID":"798ec85b-c0a7-4738-a711-d72c0c6c8b4a","Type":"ContainerStarted","Data":"ac5fc0b77539350aeff37e6f22b9fede30aa4e0c15cf8e604e598cf84ff11fa1"} Jul 14 17:27:09.904915 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:09.904485 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lglc8" event={"ID":"798ec85b-c0a7-4738-a711-d72c0c6c8b4a","Type":"ContainerStarted","Data":"3555b233b60ebf75965b2eb806330056b279098211fa1da7324b453ba0b4e3c5"} Jul 14 17:27:09.906272 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:09.906242 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c66bc44b6-9d8hc" event={"ID":"ff9d44ef-36ce-4659-89e3-5021b6fd3ce4","Type":"ContainerStarted","Data":"14742c3e7cecbe68a69975ba75d4b954d61a6864e48a27f4e34d8035c8beee2c"} Jul 14 17:27:09.927617 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:09.927556 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-lglc8" podStartSLOduration=3.9367124650000003 podStartE2EDuration="4.927535442s" totalImagesPullingTime="921.995568ms" totalInitContainerRuntime="68.827409ms" isStatefulPod=true podCreationTimestamp="2026-07-14 17:27:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:27:09.925641427 +0000 UTC m=+74.066890132" watchObservedRunningTime="2026-07-14 17:27:09.927535442 +0000 UTC m=+74.068784145" Jul 14 17:27:11.920467 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:11.920427 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-777fb8f587-jwff6" event={"ID":"ab2cd57e-c525-4003-99b4-488ca8f02e9e","Type":"ContainerStarted","Data":"3b5aea12f44481cac88d2a0f7878a38a911dfbe8b141cb433de9694ee23f0230"} Jul 14 17:27:11.920965 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:11.920517 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-777fb8f587-jwff6" event={"ID":"ab2cd57e-c525-4003-99b4-488ca8f02e9e","Type":"ContainerStarted","Data":"171af6a95abbfdb995dead988d80db6649b25803e7f8762965be209771373777"} Jul 14 17:27:11.920965 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:11.920561 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-777fb8f587-jwff6" event={"ID":"ab2cd57e-c525-4003-99b4-488ca8f02e9e","Type":"ContainerStarted","Data":"927d4818b51e230126dbfe06a7912c0adbcaa100bbf51125bde8eb0b98a6a5d9"} Jul 14 17:27:18.755896 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:18.755763 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5664c484d8-949jw"] Jul 14 17:27:18.830409 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:18.830333 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5664c484d8-949jw"] Jul 14 17:27:18.830584 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:18.830506 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5664c484d8-949jw" Jul 14 17:27:18.844094 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:18.844045 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 14 17:27:18.985962 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:18.985930 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1e6518ed-8b51-4278-a61e-11f1e81692af-service-ca\") pod \"console-5664c484d8-949jw\" (UID: \"1e6518ed-8b51-4278-a61e-11f1e81692af\") " pod="openshift-console/console-5664c484d8-949jw" Jul 14 17:27:18.986155 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:18.985971 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1e6518ed-8b51-4278-a61e-11f1e81692af-console-oauth-config\") pod \"console-5664c484d8-949jw\" (UID: \"1e6518ed-8b51-4278-a61e-11f1e81692af\") " pod="openshift-console/console-5664c484d8-949jw" Jul 14 17:27:18.986155 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:18.985992 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1e6518ed-8b51-4278-a61e-11f1e81692af-console-config\") pod \"console-5664c484d8-949jw\" (UID: \"1e6518ed-8b51-4278-a61e-11f1e81692af\") " pod="openshift-console/console-5664c484d8-949jw" Jul 14 17:27:18.986155 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:18.986061 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1e6518ed-8b51-4278-a61e-11f1e81692af-console-serving-cert\") pod \"console-5664c484d8-949jw\" (UID: \"1e6518ed-8b51-4278-a61e-11f1e81692af\") " pod="openshift-console/console-5664c484d8-949jw" Jul 14 17:27:18.986155 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:18.986107 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e6518ed-8b51-4278-a61e-11f1e81692af-trusted-ca-bundle\") pod \"console-5664c484d8-949jw\" (UID: \"1e6518ed-8b51-4278-a61e-11f1e81692af\") " pod="openshift-console/console-5664c484d8-949jw" Jul 14 17:27:18.986389 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:18.986174 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1e6518ed-8b51-4278-a61e-11f1e81692af-oauth-serving-cert\") pod \"console-5664c484d8-949jw\" (UID: \"1e6518ed-8b51-4278-a61e-11f1e81692af\") " pod="openshift-console/console-5664c484d8-949jw" Jul 14 17:27:18.986389 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:18.986244 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bfd2\" (UniqueName: \"kubernetes.io/projected/1e6518ed-8b51-4278-a61e-11f1e81692af-kube-api-access-2bfd2\") pod \"console-5664c484d8-949jw\" (UID: \"1e6518ed-8b51-4278-a61e-11f1e81692af\") " pod="openshift-console/console-5664c484d8-949jw" Jul 14 17:27:19.087588 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:19.087501 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1e6518ed-8b51-4278-a61e-11f1e81692af-console-oauth-config\") pod \"console-5664c484d8-949jw\" (UID: \"1e6518ed-8b51-4278-a61e-11f1e81692af\") " pod="openshift-console/console-5664c484d8-949jw" Jul 14 17:27:19.087588 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:19.087546 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1e6518ed-8b51-4278-a61e-11f1e81692af-console-config\") pod \"console-5664c484d8-949jw\" (UID: \"1e6518ed-8b51-4278-a61e-11f1e81692af\") " pod="openshift-console/console-5664c484d8-949jw" Jul 14 17:27:19.087588 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:19.087573 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1e6518ed-8b51-4278-a61e-11f1e81692af-console-serving-cert\") pod \"console-5664c484d8-949jw\" (UID: \"1e6518ed-8b51-4278-a61e-11f1e81692af\") " pod="openshift-console/console-5664c484d8-949jw" Jul 14 17:27:19.087880 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:19.087601 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e6518ed-8b51-4278-a61e-11f1e81692af-trusted-ca-bundle\") pod \"console-5664c484d8-949jw\" (UID: \"1e6518ed-8b51-4278-a61e-11f1e81692af\") " pod="openshift-console/console-5664c484d8-949jw" Jul 14 17:27:19.087880 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:19.087665 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1e6518ed-8b51-4278-a61e-11f1e81692af-oauth-serving-cert\") pod \"console-5664c484d8-949jw\" (UID: \"1e6518ed-8b51-4278-a61e-11f1e81692af\") " pod="openshift-console/console-5664c484d8-949jw" Jul 14 17:27:19.087880 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:19.087707 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bfd2\" (UniqueName: \"kubernetes.io/projected/1e6518ed-8b51-4278-a61e-11f1e81692af-kube-api-access-2bfd2\") pod \"console-5664c484d8-949jw\" (UID: \"1e6518ed-8b51-4278-a61e-11f1e81692af\") " pod="openshift-console/console-5664c484d8-949jw" Jul 14 17:27:19.087880 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:19.087734 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1e6518ed-8b51-4278-a61e-11f1e81692af-service-ca\") pod \"console-5664c484d8-949jw\" (UID: \"1e6518ed-8b51-4278-a61e-11f1e81692af\") " pod="openshift-console/console-5664c484d8-949jw" Jul 14 17:27:19.088490 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:19.088462 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1e6518ed-8b51-4278-a61e-11f1e81692af-console-config\") pod \"console-5664c484d8-949jw\" (UID: \"1e6518ed-8b51-4278-a61e-11f1e81692af\") " pod="openshift-console/console-5664c484d8-949jw" Jul 14 17:27:19.088658 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:19.088637 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1e6518ed-8b51-4278-a61e-11f1e81692af-oauth-serving-cert\") pod \"console-5664c484d8-949jw\" (UID: \"1e6518ed-8b51-4278-a61e-11f1e81692af\") " pod="openshift-console/console-5664c484d8-949jw" Jul 14 17:27:19.088731 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:19.088662 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1e6518ed-8b51-4278-a61e-11f1e81692af-service-ca\") pod \"console-5664c484d8-949jw\" (UID: \"1e6518ed-8b51-4278-a61e-11f1e81692af\") " pod="openshift-console/console-5664c484d8-949jw" Jul 14 17:27:19.088973 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:19.088951 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e6518ed-8b51-4278-a61e-11f1e81692af-trusted-ca-bundle\") pod \"console-5664c484d8-949jw\" (UID: \"1e6518ed-8b51-4278-a61e-11f1e81692af\") " pod="openshift-console/console-5664c484d8-949jw" Jul 14 17:27:19.090254 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:19.090233 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1e6518ed-8b51-4278-a61e-11f1e81692af-console-oauth-config\") pod \"console-5664c484d8-949jw\" (UID: \"1e6518ed-8b51-4278-a61e-11f1e81692af\") " pod="openshift-console/console-5664c484d8-949jw" Jul 14 17:27:19.090484 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:19.090457 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1e6518ed-8b51-4278-a61e-11f1e81692af-console-serving-cert\") pod \"console-5664c484d8-949jw\" (UID: \"1e6518ed-8b51-4278-a61e-11f1e81692af\") " pod="openshift-console/console-5664c484d8-949jw" Jul 14 17:27:19.096864 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:19.096845 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bfd2\" (UniqueName: \"kubernetes.io/projected/1e6518ed-8b51-4278-a61e-11f1e81692af-kube-api-access-2bfd2\") pod \"console-5664c484d8-949jw\" (UID: \"1e6518ed-8b51-4278-a61e-11f1e81692af\") " pod="openshift-console/console-5664c484d8-949jw" Jul 14 17:27:19.144367 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:19.144336 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5664c484d8-949jw" Jul 14 17:27:20.487441 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:20.487366 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5664c484d8-949jw"] Jul 14 17:27:20.489257 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:27:20.489224 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e6518ed_8b51_4278_a61e_11f1e81692af.slice/crio-25c56403d10d164658b5e5cbb302f06b7f00d71a7dfad9af27990425f461b829 WatchSource:0}: Error finding container 25c56403d10d164658b5e5cbb302f06b7f00d71a7dfad9af27990425f461b829: Status 404 returned error can't find the container with id 25c56403d10d164658b5e5cbb302f06b7f00d71a7dfad9af27990425f461b829 Jul 14 17:27:20.959779 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:20.959544 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-777fb8f587-jwff6" event={"ID":"ab2cd57e-c525-4003-99b4-488ca8f02e9e","Type":"ContainerStarted","Data":"12505f4dc1ab7c7556a7314dab63c8e230626daf13ad4481cf5b6573ef787fb6"} Jul 14 17:27:20.959779 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:20.959615 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-777fb8f587-jwff6" event={"ID":"ab2cd57e-c525-4003-99b4-488ca8f02e9e","Type":"ContainerStarted","Data":"9c5204a0530c46020cd4fc488cd7b7a71a55046643056b4828e6c0a1e6d6267f"} Jul 14 17:27:20.961515 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:20.961489 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5664c484d8-949jw" event={"ID":"1e6518ed-8b51-4278-a61e-11f1e81692af","Type":"ContainerStarted","Data":"c07c4997c9b7beb4f7b05f0ffd6bc57b5c7468154dd7e794ccb5e269bbee797c"} Jul 14 17:27:20.961622 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:20.961538 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5664c484d8-949jw" event={"ID":"1e6518ed-8b51-4278-a61e-11f1e81692af","Type":"ContainerStarted","Data":"25c56403d10d164658b5e5cbb302f06b7f00d71a7dfad9af27990425f461b829"} Jul 14 17:27:20.963496 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:20.963417 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-vgj5f" event={"ID":"9415d694-c5e5-492d-a2f2-348323c40cd7","Type":"ContainerStarted","Data":"f2ecd13f3dc3db6d7dc810d81f9f926ffd308078d2f2944502ce720156d96a62"} Jul 14 17:27:20.963598 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:20.963572 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-vgj5f" Jul 14 17:27:20.966136 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:20.966117 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c66bc44b6-9d8hc" event={"ID":"ff9d44ef-36ce-4659-89e3-5021b6fd3ce4","Type":"ContainerStarted","Data":"f888bb856b611e45e0ba6f3761e1006063c4056aba00d21567310e285a59e4c4"} Jul 14 17:27:20.969793 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:20.969774 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-vgj5f" Jul 14 17:27:20.980189 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:20.980105 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5664c484d8-949jw" podStartSLOduration=2.980085229 podStartE2EDuration="2.980085229s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:27:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:27:20.978542744 +0000 UTC m=+85.119791448" watchObservedRunningTime="2026-07-14 17:27:20.980085229 +0000 UTC m=+85.121333932" Jul 14 17:27:20.994309 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:20.994263 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-vgj5f" podStartSLOduration=1.071980593 podStartE2EDuration="22.994249341s" totalImagesPullingTime="21.922268748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:26:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:27:20.993532702 +0000 UTC m=+85.134781432" watchObservedRunningTime="2026-07-14 17:27:20.994249341 +0000 UTC m=+85.135498045" Jul 14 17:27:21.012993 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:21.012933 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c66bc44b6-9d8hc" podStartSLOduration=2.148836814 podStartE2EDuration="13.012912043s" totalImagesPullingTime="10.864075229s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:27:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:27:21.011082683 +0000 UTC m=+85.152331387" watchObservedRunningTime="2026-07-14 17:27:21.012912043 +0000 UTC m=+85.154160748" Jul 14 17:27:21.974806 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:21.974749 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-777fb8f587-jwff6" event={"ID":"ab2cd57e-c525-4003-99b4-488ca8f02e9e","Type":"ContainerStarted","Data":"9f72d2f6a11272c4184607f3caa340f0c65d670ed19dbd3eb887416016eb7b3e"} Jul 14 17:27:21.998840 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:21.998777 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-777fb8f587-jwff6" podStartSLOduration=2.431530748 podStartE2EDuration="14.998756933s" totalImagesPullingTime="12.567226185s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:27:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:27:21.998493662 +0000 UTC m=+86.139742400" watchObservedRunningTime="2026-07-14 17:27:21.998756933 +0000 UTC m=+86.140005640" Jul 14 17:27:22.979257 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:22.979226 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-777fb8f587-jwff6" Jul 14 17:27:22.985093 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:22.985067 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-777fb8f587-jwff6" Jul 14 17:27:29.145375 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:29.145326 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5664c484d8-949jw" Jul 14 17:27:29.145849 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:29.145399 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5664c484d8-949jw" Jul 14 17:27:29.150369 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:29.150347 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5664c484d8-949jw" Jul 14 17:27:29.274839 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:29.274799 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c66bc44b6-9d8hc" Jul 14 17:27:29.275014 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:29.274869 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c66bc44b6-9d8hc" Jul 14 17:27:29.279612 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:29.279588 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c66bc44b6-9d8hc" Jul 14 17:27:30.021613 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:30.021587 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5664c484d8-949jw" Jul 14 17:27:30.022089 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:30.022066 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c66bc44b6-9d8hc" Jul 14 17:27:30.087200 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:30.087166 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c66bc44b6-9d8hc"] Jul 14 17:27:38.900780 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:38.900745 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-4lhqc" Jul 14 17:27:57.047162 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:57.047113 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7c66bc44b6-9d8hc" podUID="ff9d44ef-36ce-4659-89e3-5021b6fd3ce4" containerName="console" containerID="cri-o://f888bb856b611e45e0ba6f3761e1006063c4056aba00d21567310e285a59e4c4" gracePeriod=15 Jul 14 17:27:57.325406 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:57.325345 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c66bc44b6-9d8hc" Jul 14 17:27:57.396695 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:57.396664 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ff9d44ef-36ce-4659-89e3-5021b6fd3ce4-oauth-serving-cert\") pod \"ff9d44ef-36ce-4659-89e3-5021b6fd3ce4\" (UID: \"ff9d44ef-36ce-4659-89e3-5021b6fd3ce4\") " Jul 14 17:27:57.396875 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:57.396712 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ff9d44ef-36ce-4659-89e3-5021b6fd3ce4-console-config\") pod \"ff9d44ef-36ce-4659-89e3-5021b6fd3ce4\" (UID: \"ff9d44ef-36ce-4659-89e3-5021b6fd3ce4\") " Jul 14 17:27:57.396875 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:57.396736 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sv4ph\" (UniqueName: \"kubernetes.io/projected/ff9d44ef-36ce-4659-89e3-5021b6fd3ce4-kube-api-access-sv4ph\") pod \"ff9d44ef-36ce-4659-89e3-5021b6fd3ce4\" (UID: \"ff9d44ef-36ce-4659-89e3-5021b6fd3ce4\") " Jul 14 17:27:57.396875 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:57.396774 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ff9d44ef-36ce-4659-89e3-5021b6fd3ce4-console-oauth-config\") pod \"ff9d44ef-36ce-4659-89e3-5021b6fd3ce4\" (UID: \"ff9d44ef-36ce-4659-89e3-5021b6fd3ce4\") " Jul 14 17:27:57.396875 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:57.396801 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff9d44ef-36ce-4659-89e3-5021b6fd3ce4-console-serving-cert\") pod \"ff9d44ef-36ce-4659-89e3-5021b6fd3ce4\" (UID: \"ff9d44ef-36ce-4659-89e3-5021b6fd3ce4\") " Jul 14 17:27:57.396875 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:57.396840 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ff9d44ef-36ce-4659-89e3-5021b6fd3ce4-service-ca\") pod \"ff9d44ef-36ce-4659-89e3-5021b6fd3ce4\" (UID: \"ff9d44ef-36ce-4659-89e3-5021b6fd3ce4\") " Jul 14 17:27:57.397200 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:57.397161 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff9d44ef-36ce-4659-89e3-5021b6fd3ce4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ff9d44ef-36ce-4659-89e3-5021b6fd3ce4" (UID: "ff9d44ef-36ce-4659-89e3-5021b6fd3ce4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:27:57.397366 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:57.397172 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff9d44ef-36ce-4659-89e3-5021b6fd3ce4-console-config" (OuterVolumeSpecName: "console-config") pod "ff9d44ef-36ce-4659-89e3-5021b6fd3ce4" (UID: "ff9d44ef-36ce-4659-89e3-5021b6fd3ce4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:27:57.397366 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:57.397306 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff9d44ef-36ce-4659-89e3-5021b6fd3ce4-service-ca" (OuterVolumeSpecName: "service-ca") pod "ff9d44ef-36ce-4659-89e3-5021b6fd3ce4" (UID: "ff9d44ef-36ce-4659-89e3-5021b6fd3ce4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:27:57.399072 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:57.399042 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff9d44ef-36ce-4659-89e3-5021b6fd3ce4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ff9d44ef-36ce-4659-89e3-5021b6fd3ce4" (UID: "ff9d44ef-36ce-4659-89e3-5021b6fd3ce4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:27:57.399170 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:57.399045 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff9d44ef-36ce-4659-89e3-5021b6fd3ce4-kube-api-access-sv4ph" (OuterVolumeSpecName: "kube-api-access-sv4ph") pod "ff9d44ef-36ce-4659-89e3-5021b6fd3ce4" (UID: "ff9d44ef-36ce-4659-89e3-5021b6fd3ce4"). InnerVolumeSpecName "kube-api-access-sv4ph". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:27:57.399170 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:57.399054 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff9d44ef-36ce-4659-89e3-5021b6fd3ce4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ff9d44ef-36ce-4659-89e3-5021b6fd3ce4" (UID: "ff9d44ef-36ce-4659-89e3-5021b6fd3ce4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:27:57.498470 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:57.498435 2554 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ff9d44ef-36ce-4659-89e3-5021b6fd3ce4-console-oauth-config\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:27:57.498470 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:57.498462 2554 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff9d44ef-36ce-4659-89e3-5021b6fd3ce4-console-serving-cert\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:27:57.498470 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:57.498473 2554 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ff9d44ef-36ce-4659-89e3-5021b6fd3ce4-service-ca\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:27:57.498683 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:57.498484 2554 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ff9d44ef-36ce-4659-89e3-5021b6fd3ce4-oauth-serving-cert\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:27:57.498683 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:57.498492 2554 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ff9d44ef-36ce-4659-89e3-5021b6fd3ce4-console-config\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:27:57.498683 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:57.498501 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sv4ph\" (UniqueName: \"kubernetes.io/projected/ff9d44ef-36ce-4659-89e3-5021b6fd3ce4-kube-api-access-sv4ph\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:27:58.122490 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:58.122455 2554 generic.go:356] "Generic (PLEG): container finished" podID="ff9d44ef-36ce-4659-89e3-5021b6fd3ce4" containerID="f888bb856b611e45e0ba6f3761e1006063c4056aba00d21567310e285a59e4c4" exitCode=0 Jul 14 17:27:58.122863 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:58.122499 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c66bc44b6-9d8hc" Jul 14 17:27:58.122863 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:58.122530 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c66bc44b6-9d8hc" event={"ID":"ff9d44ef-36ce-4659-89e3-5021b6fd3ce4","Type":"ContainerDied","Data":"f888bb856b611e45e0ba6f3761e1006063c4056aba00d21567310e285a59e4c4"} Jul 14 17:27:58.122863 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:58.122574 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c66bc44b6-9d8hc" event={"ID":"ff9d44ef-36ce-4659-89e3-5021b6fd3ce4","Type":"ContainerDied","Data":"14742c3e7cecbe68a69975ba75d4b954d61a6864e48a27f4e34d8035c8beee2c"} Jul 14 17:27:58.122863 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:58.122597 2554 scope.go:117] "RemoveContainer" containerID="f888bb856b611e45e0ba6f3761e1006063c4056aba00d21567310e285a59e4c4" Jul 14 17:27:58.137407 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:58.137379 2554 scope.go:117] "RemoveContainer" containerID="f888bb856b611e45e0ba6f3761e1006063c4056aba00d21567310e285a59e4c4" Jul 14 17:27:58.137787 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:27:58.137762 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f888bb856b611e45e0ba6f3761e1006063c4056aba00d21567310e285a59e4c4\": container with ID starting with f888bb856b611e45e0ba6f3761e1006063c4056aba00d21567310e285a59e4c4 not found: ID does not exist" containerID="f888bb856b611e45e0ba6f3761e1006063c4056aba00d21567310e285a59e4c4" Jul 14 17:27:58.137847 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:58.137801 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f888bb856b611e45e0ba6f3761e1006063c4056aba00d21567310e285a59e4c4"} err="failed to get container status \"f888bb856b611e45e0ba6f3761e1006063c4056aba00d21567310e285a59e4c4\": rpc error: code = NotFound desc = could not find container \"f888bb856b611e45e0ba6f3761e1006063c4056aba00d21567310e285a59e4c4\": container with ID starting with f888bb856b611e45e0ba6f3761e1006063c4056aba00d21567310e285a59e4c4 not found: ID does not exist" Jul 14 17:27:58.146528 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:58.146507 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c66bc44b6-9d8hc"] Jul 14 17:27:58.150062 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:58.150040 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c66bc44b6-9d8hc"] Jul 14 17:27:58.386410 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:27:58.386336 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff9d44ef-36ce-4659-89e3-5021b6fd3ce4" path="/var/lib/kubelet/pods/ff9d44ef-36ce-4659-89e3-5021b6fd3ce4/volumes" Jul 14 17:28:27.761821 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:27.761787 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 17:28:27.762308 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:27.762031 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff9d44ef-36ce-4659-89e3-5021b6fd3ce4" containerName="console" Jul 14 17:28:27.762308 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:27.762042 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff9d44ef-36ce-4659-89e3-5021b6fd3ce4" containerName="console" Jul 14 17:28:27.762308 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:27.762097 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff9d44ef-36ce-4659-89e3-5021b6fd3ce4" containerName="console" Jul 14 17:28:27.764390 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:27.764358 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:27.767205 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:27.767179 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 14 17:28:27.767334 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:27.767241 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 14 17:28:27.767334 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:27.767317 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 14 17:28:27.767459 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:27.767335 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 14 17:28:27.767459 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:27.767413 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-klxtj\"" Jul 14 17:28:27.767541 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:27.767468 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 14 17:28:27.767605 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:27.767592 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 14 17:28:27.767653 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:27.767639 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 14 17:28:27.767983 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:27.767961 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 14 17:28:27.775625 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:27.775607 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 14 17:28:27.778558 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:27.778535 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 17:28:27.921028 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:27.920992 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14bf1b4f-bd69-4e71-af6e-8eb73d1c7087-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:27.921028 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:27.921027 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/14bf1b4f-bd69-4e71-af6e-8eb73d1c7087-web-config\") pod \"alertmanager-main-0\" (UID: \"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:27.921200 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:27.921047 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/14bf1b4f-bd69-4e71-af6e-8eb73d1c7087-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:27.921200 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:27.921065 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/14bf1b4f-bd69-4e71-af6e-8eb73d1c7087-config-volume\") pod \"alertmanager-main-0\" (UID: \"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:27.921200 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:27.921153 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/14bf1b4f-bd69-4e71-af6e-8eb73d1c7087-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:27.921200 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:27.921191 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/14bf1b4f-bd69-4e71-af6e-8eb73d1c7087-tls-assets\") pod \"alertmanager-main-0\" (UID: \"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:27.921359 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:27.921224 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/14bf1b4f-bd69-4e71-af6e-8eb73d1c7087-config-out\") pod \"alertmanager-main-0\" (UID: \"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:27.921359 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:27.921248 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/14bf1b4f-bd69-4e71-af6e-8eb73d1c7087-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:27.921359 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:27.921283 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/14bf1b4f-bd69-4e71-af6e-8eb73d1c7087-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:27.921359 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:27.921304 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/14bf1b4f-bd69-4e71-af6e-8eb73d1c7087-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:27.921359 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:27.921321 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thjrj\" (UniqueName: \"kubernetes.io/projected/14bf1b4f-bd69-4e71-af6e-8eb73d1c7087-kube-api-access-thjrj\") pod \"alertmanager-main-0\" (UID: \"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:27.921529 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:27.921359 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/14bf1b4f-bd69-4e71-af6e-8eb73d1c7087-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:27.921529 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:27.921378 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/14bf1b4f-bd69-4e71-af6e-8eb73d1c7087-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:28.022691 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:28.022619 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thjrj\" (UniqueName: \"kubernetes.io/projected/14bf1b4f-bd69-4e71-af6e-8eb73d1c7087-kube-api-access-thjrj\") pod \"alertmanager-main-0\" (UID: \"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:28.022691 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:28.022653 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/14bf1b4f-bd69-4e71-af6e-8eb73d1c7087-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:28.022691 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:28.022672 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/14bf1b4f-bd69-4e71-af6e-8eb73d1c7087-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:28.022951 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:28.022803 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14bf1b4f-bd69-4e71-af6e-8eb73d1c7087-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:28.022951 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:28.022839 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/14bf1b4f-bd69-4e71-af6e-8eb73d1c7087-web-config\") pod \"alertmanager-main-0\" (UID: \"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:28.022951 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:28.022874 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/14bf1b4f-bd69-4e71-af6e-8eb73d1c7087-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:28.022951 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:28.022897 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/14bf1b4f-bd69-4e71-af6e-8eb73d1c7087-config-volume\") pod \"alertmanager-main-0\" (UID: \"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:28.022951 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:28.022931 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/14bf1b4f-bd69-4e71-af6e-8eb73d1c7087-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:28.023206 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:28.022957 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/14bf1b4f-bd69-4e71-af6e-8eb73d1c7087-tls-assets\") pod \"alertmanager-main-0\" (UID: \"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:28.023206 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:28.022979 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/14bf1b4f-bd69-4e71-af6e-8eb73d1c7087-config-out\") pod \"alertmanager-main-0\" (UID: \"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:28.023206 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:28.023003 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/14bf1b4f-bd69-4e71-af6e-8eb73d1c7087-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:28.023206 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:28.023048 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/14bf1b4f-bd69-4e71-af6e-8eb73d1c7087-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:28.023206 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:28.023079 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/14bf1b4f-bd69-4e71-af6e-8eb73d1c7087-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:28.023484 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:28.023358 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/14bf1b4f-bd69-4e71-af6e-8eb73d1c7087-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:28.023810 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:28.023779 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/14bf1b4f-bd69-4e71-af6e-8eb73d1c7087-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:28.024050 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:28.024015 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14bf1b4f-bd69-4e71-af6e-8eb73d1c7087-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:28.026139 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:28.026107 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/14bf1b4f-bd69-4e71-af6e-8eb73d1c7087-web-config\") pod \"alertmanager-main-0\" (UID: \"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:28.026259 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:28.026116 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/14bf1b4f-bd69-4e71-af6e-8eb73d1c7087-config-out\") pod \"alertmanager-main-0\" (UID: \"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:28.026328 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:28.026309 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/14bf1b4f-bd69-4e71-af6e-8eb73d1c7087-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:28.026388 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:28.026330 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/14bf1b4f-bd69-4e71-af6e-8eb73d1c7087-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:28.026516 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:28.026496 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/14bf1b4f-bd69-4e71-af6e-8eb73d1c7087-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:28.026572 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:28.026557 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/14bf1b4f-bd69-4e71-af6e-8eb73d1c7087-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:28.027192 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:28.027174 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/14bf1b4f-bd69-4e71-af6e-8eb73d1c7087-config-volume\") pod \"alertmanager-main-0\" (UID: \"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:28.027617 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:28.027588 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/14bf1b4f-bd69-4e71-af6e-8eb73d1c7087-tls-assets\") pod \"alertmanager-main-0\" (UID: \"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:28.027785 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:28.027764 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/14bf1b4f-bd69-4e71-af6e-8eb73d1c7087-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:28.031360 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:28.031332 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thjrj\" (UniqueName: \"kubernetes.io/projected/14bf1b4f-bd69-4e71-af6e-8eb73d1c7087-kube-api-access-thjrj\") pod \"alertmanager-main-0\" (UID: \"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:28.077515 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:28.077482 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:28:28.214398 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:28.214363 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 17:28:28.215468 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:28:28.215436 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14bf1b4f_bd69_4e71_af6e_8eb73d1c7087.slice/crio-6135622f38bfdf4a14772c573d2fe50dba74c4f26b0a5300139fc10bde3ae18e WatchSource:0}: Error finding container 6135622f38bfdf4a14772c573d2fe50dba74c4f26b0a5300139fc10bde3ae18e: Status 404 returned error can't find the container with id 6135622f38bfdf4a14772c573d2fe50dba74c4f26b0a5300139fc10bde3ae18e Jul 14 17:28:28.226207 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:28.226181 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087","Type":"ContainerStarted","Data":"6135622f38bfdf4a14772c573d2fe50dba74c4f26b0a5300139fc10bde3ae18e"} Jul 14 17:28:30.234811 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:30.234777 2554 generic.go:356] "Generic (PLEG): container finished" podID="14bf1b4f-bd69-4e71-af6e-8eb73d1c7087" containerID="e353e1e0acdbdacdfd1dd25d42c7748898ea2184f5f5e6e0933385c38f7116d2" exitCode=0 Jul 14 17:28:30.235244 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:30.234851 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087","Type":"ContainerDied","Data":"e353e1e0acdbdacdfd1dd25d42c7748898ea2184f5f5e6e0933385c38f7116d2"} Jul 14 17:28:32.245675 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:32.245589 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087","Type":"ContainerStarted","Data":"91dc43688facc936583c13d4498e35fee5b09a5905ab51326bb2ababa27aa73f"} Jul 14 17:28:32.246008 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:32.245703 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087","Type":"ContainerStarted","Data":"2721c8894fcd4e61ffe248f6d7af94a9dd9be19dd698d787bf31422c59b578c8"} Jul 14 17:28:32.246008 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:32.245748 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087","Type":"ContainerStarted","Data":"c8946d74c5d34cfd9571a309b363b09b2dcc5fd5a57d9abad9ff9ed5b9159265"} Jul 14 17:28:32.246008 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:32.245782 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087","Type":"ContainerStarted","Data":"06d67596c74bbd46ae83024236a2340976f5f1e8c4730986d0d45f3efdf31c9f"} Jul 14 17:28:32.246008 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:32.245818 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087","Type":"ContainerStarted","Data":"9600d5b7eb2f4d1eac39ef6affc20e00738cf93fae8065dd7033f478242994e0"} Jul 14 17:28:32.246008 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:32.245852 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"14bf1b4f-bd69-4e71-af6e-8eb73d1c7087","Type":"ContainerStarted","Data":"f4a85d522385e97ad5c4b1a1855a9bd9769472671f41d3f1f2058f66935c47fd"} Jul 14 17:28:32.272156 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:32.272114 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.360957512 podStartE2EDuration="5.272099712s" totalImagesPullingTime="2.895163578s" totalInitContainerRuntime="15.978622ms" isStatefulPod=false podCreationTimestamp="2026-07-14 17:28:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:28:32.271535945 +0000 UTC m=+156.412784652" watchObservedRunningTime="2026-07-14 17:28:32.272099712 +0000 UTC m=+156.413348417" Jul 14 17:28:34.459696 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:34.459662 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-c6dfccffc-dj7fd"] Jul 14 17:28:34.461847 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:34.461830 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c6dfccffc-dj7fd" Jul 14 17:28:34.477752 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:34.475262 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c6dfccffc-dj7fd"] Jul 14 17:28:34.578065 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:34.578027 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af02cb8d-c30f-4e48-9bd9-404750d96f82-console-config\") pod \"console-c6dfccffc-dj7fd\" (UID: \"af02cb8d-c30f-4e48-9bd9-404750d96f82\") " pod="openshift-console/console-c6dfccffc-dj7fd" Jul 14 17:28:34.578065 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:34.578076 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af02cb8d-c30f-4e48-9bd9-404750d96f82-oauth-serving-cert\") pod \"console-c6dfccffc-dj7fd\" (UID: \"af02cb8d-c30f-4e48-9bd9-404750d96f82\") " pod="openshift-console/console-c6dfccffc-dj7fd" Jul 14 17:28:34.578358 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:34.578132 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af02cb8d-c30f-4e48-9bd9-404750d96f82-service-ca\") pod \"console-c6dfccffc-dj7fd\" (UID: \"af02cb8d-c30f-4e48-9bd9-404750d96f82\") " pod="openshift-console/console-c6dfccffc-dj7fd" Jul 14 17:28:34.578358 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:34.578172 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af02cb8d-c30f-4e48-9bd9-404750d96f82-console-serving-cert\") pod \"console-c6dfccffc-dj7fd\" (UID: \"af02cb8d-c30f-4e48-9bd9-404750d96f82\") " pod="openshift-console/console-c6dfccffc-dj7fd" Jul 14 17:28:34.578358 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:34.578232 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af02cb8d-c30f-4e48-9bd9-404750d96f82-console-oauth-config\") pod \"console-c6dfccffc-dj7fd\" (UID: \"af02cb8d-c30f-4e48-9bd9-404750d96f82\") " pod="openshift-console/console-c6dfccffc-dj7fd" Jul 14 17:28:34.578358 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:34.578261 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af02cb8d-c30f-4e48-9bd9-404750d96f82-trusted-ca-bundle\") pod \"console-c6dfccffc-dj7fd\" (UID: \"af02cb8d-c30f-4e48-9bd9-404750d96f82\") " pod="openshift-console/console-c6dfccffc-dj7fd" Jul 14 17:28:34.578358 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:34.578295 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tk6vd\" (UniqueName: \"kubernetes.io/projected/af02cb8d-c30f-4e48-9bd9-404750d96f82-kube-api-access-tk6vd\") pod \"console-c6dfccffc-dj7fd\" (UID: \"af02cb8d-c30f-4e48-9bd9-404750d96f82\") " pod="openshift-console/console-c6dfccffc-dj7fd" Jul 14 17:28:34.678896 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:34.678863 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af02cb8d-c30f-4e48-9bd9-404750d96f82-console-serving-cert\") pod \"console-c6dfccffc-dj7fd\" (UID: \"af02cb8d-c30f-4e48-9bd9-404750d96f82\") " pod="openshift-console/console-c6dfccffc-dj7fd" Jul 14 17:28:34.679047 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:34.678914 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af02cb8d-c30f-4e48-9bd9-404750d96f82-console-oauth-config\") pod \"console-c6dfccffc-dj7fd\" (UID: \"af02cb8d-c30f-4e48-9bd9-404750d96f82\") " pod="openshift-console/console-c6dfccffc-dj7fd" Jul 14 17:28:34.679047 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:34.678944 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af02cb8d-c30f-4e48-9bd9-404750d96f82-trusted-ca-bundle\") pod \"console-c6dfccffc-dj7fd\" (UID: \"af02cb8d-c30f-4e48-9bd9-404750d96f82\") " pod="openshift-console/console-c6dfccffc-dj7fd" Jul 14 17:28:34.679047 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:34.678971 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tk6vd\" (UniqueName: \"kubernetes.io/projected/af02cb8d-c30f-4e48-9bd9-404750d96f82-kube-api-access-tk6vd\") pod \"console-c6dfccffc-dj7fd\" (UID: \"af02cb8d-c30f-4e48-9bd9-404750d96f82\") " pod="openshift-console/console-c6dfccffc-dj7fd" Jul 14 17:28:34.679047 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:34.679012 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af02cb8d-c30f-4e48-9bd9-404750d96f82-console-config\") pod \"console-c6dfccffc-dj7fd\" (UID: \"af02cb8d-c30f-4e48-9bd9-404750d96f82\") " pod="openshift-console/console-c6dfccffc-dj7fd" Jul 14 17:28:34.679047 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:34.679038 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af02cb8d-c30f-4e48-9bd9-404750d96f82-oauth-serving-cert\") pod \"console-c6dfccffc-dj7fd\" (UID: \"af02cb8d-c30f-4e48-9bd9-404750d96f82\") " pod="openshift-console/console-c6dfccffc-dj7fd" Jul 14 17:28:34.679319 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:34.679082 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af02cb8d-c30f-4e48-9bd9-404750d96f82-service-ca\") pod \"console-c6dfccffc-dj7fd\" (UID: \"af02cb8d-c30f-4e48-9bd9-404750d96f82\") " pod="openshift-console/console-c6dfccffc-dj7fd" Jul 14 17:28:34.679872 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:34.679841 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af02cb8d-c30f-4e48-9bd9-404750d96f82-oauth-serving-cert\") pod \"console-c6dfccffc-dj7fd\" (UID: \"af02cb8d-c30f-4e48-9bd9-404750d96f82\") " pod="openshift-console/console-c6dfccffc-dj7fd" Jul 14 17:28:34.680009 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:34.679841 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af02cb8d-c30f-4e48-9bd9-404750d96f82-console-config\") pod \"console-c6dfccffc-dj7fd\" (UID: \"af02cb8d-c30f-4e48-9bd9-404750d96f82\") " pod="openshift-console/console-c6dfccffc-dj7fd" Jul 14 17:28:34.680009 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:34.679847 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af02cb8d-c30f-4e48-9bd9-404750d96f82-service-ca\") pod \"console-c6dfccffc-dj7fd\" (UID: \"af02cb8d-c30f-4e48-9bd9-404750d96f82\") " pod="openshift-console/console-c6dfccffc-dj7fd" Jul 14 17:28:34.680182 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:34.680156 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af02cb8d-c30f-4e48-9bd9-404750d96f82-trusted-ca-bundle\") pod \"console-c6dfccffc-dj7fd\" (UID: \"af02cb8d-c30f-4e48-9bd9-404750d96f82\") " pod="openshift-console/console-c6dfccffc-dj7fd" Jul 14 17:28:34.681456 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:34.681424 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af02cb8d-c30f-4e48-9bd9-404750d96f82-console-oauth-config\") pod \"console-c6dfccffc-dj7fd\" (UID: \"af02cb8d-c30f-4e48-9bd9-404750d96f82\") " pod="openshift-console/console-c6dfccffc-dj7fd" Jul 14 17:28:34.681606 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:34.681588 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af02cb8d-c30f-4e48-9bd9-404750d96f82-console-serving-cert\") pod \"console-c6dfccffc-dj7fd\" (UID: \"af02cb8d-c30f-4e48-9bd9-404750d96f82\") " pod="openshift-console/console-c6dfccffc-dj7fd" Jul 14 17:28:34.688112 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:34.688090 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tk6vd\" (UniqueName: \"kubernetes.io/projected/af02cb8d-c30f-4e48-9bd9-404750d96f82-kube-api-access-tk6vd\") pod \"console-c6dfccffc-dj7fd\" (UID: \"af02cb8d-c30f-4e48-9bd9-404750d96f82\") " pod="openshift-console/console-c6dfccffc-dj7fd" Jul 14 17:28:34.771051 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:34.770966 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c6dfccffc-dj7fd" Jul 14 17:28:34.892457 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:34.892433 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c6dfccffc-dj7fd"] Jul 14 17:28:34.894527 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:28:34.894496 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf02cb8d_c30f_4e48_9bd9_404750d96f82.slice/crio-729fe255d48d778e5f8a46dc514f7ae5becee0a362752b96e48aac90b85a1ce5 WatchSource:0}: Error finding container 729fe255d48d778e5f8a46dc514f7ae5becee0a362752b96e48aac90b85a1ce5: Status 404 returned error can't find the container with id 729fe255d48d778e5f8a46dc514f7ae5becee0a362752b96e48aac90b85a1ce5 Jul 14 17:28:35.258941 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:35.258907 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c6dfccffc-dj7fd" event={"ID":"af02cb8d-c30f-4e48-9bd9-404750d96f82","Type":"ContainerStarted","Data":"e330b36d775cd7b3cdf0e8d1ee8c306a2b1b22acb6af99d03acdfbe34b281e35"} Jul 14 17:28:35.259121 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:35.258957 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c6dfccffc-dj7fd" event={"ID":"af02cb8d-c30f-4e48-9bd9-404750d96f82","Type":"ContainerStarted","Data":"729fe255d48d778e5f8a46dc514f7ae5becee0a362752b96e48aac90b85a1ce5"} Jul 14 17:28:44.771355 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:44.771316 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-c6dfccffc-dj7fd" Jul 14 17:28:44.771743 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:44.771386 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c6dfccffc-dj7fd" Jul 14 17:28:44.775998 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:44.775975 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-c6dfccffc-dj7fd" Jul 14 17:28:44.794336 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:44.794298 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-c6dfccffc-dj7fd" podStartSLOduration=10.794285539 podStartE2EDuration="10.794285539s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:28:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:28:35.277837853 +0000 UTC m=+159.419086556" watchObservedRunningTime="2026-07-14 17:28:44.794285539 +0000 UTC m=+168.935534242" Jul 14 17:28:45.300782 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:45.300753 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-c6dfccffc-dj7fd" Jul 14 17:28:45.341052 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:28:45.341021 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5664c484d8-949jw"] Jul 14 17:29:10.361686 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:10.361639 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5664c484d8-949jw" podUID="1e6518ed-8b51-4278-a61e-11f1e81692af" containerName="console" containerID="cri-o://c07c4997c9b7beb4f7b05f0ffd6bc57b5c7468154dd7e794ccb5e269bbee797c" gracePeriod=15 Jul 14 17:29:10.592390 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:10.592367 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5664c484d8-949jw" Jul 14 17:29:10.655599 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:10.655532 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1e6518ed-8b51-4278-a61e-11f1e81692af-console-serving-cert\") pod \"1e6518ed-8b51-4278-a61e-11f1e81692af\" (UID: \"1e6518ed-8b51-4278-a61e-11f1e81692af\") " Jul 14 17:29:10.655599 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:10.655578 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1e6518ed-8b51-4278-a61e-11f1e81692af-console-oauth-config\") pod \"1e6518ed-8b51-4278-a61e-11f1e81692af\" (UID: \"1e6518ed-8b51-4278-a61e-11f1e81692af\") " Jul 14 17:29:10.655803 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:10.655613 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1e6518ed-8b51-4278-a61e-11f1e81692af-console-config\") pod \"1e6518ed-8b51-4278-a61e-11f1e81692af\" (UID: \"1e6518ed-8b51-4278-a61e-11f1e81692af\") " Jul 14 17:29:10.655803 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:10.655637 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e6518ed-8b51-4278-a61e-11f1e81692af-trusted-ca-bundle\") pod \"1e6518ed-8b51-4278-a61e-11f1e81692af\" (UID: \"1e6518ed-8b51-4278-a61e-11f1e81692af\") " Jul 14 17:29:10.655803 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:10.655657 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2bfd2\" (UniqueName: \"kubernetes.io/projected/1e6518ed-8b51-4278-a61e-11f1e81692af-kube-api-access-2bfd2\") pod \"1e6518ed-8b51-4278-a61e-11f1e81692af\" (UID: \"1e6518ed-8b51-4278-a61e-11f1e81692af\") " Jul 14 17:29:10.655803 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:10.655699 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1e6518ed-8b51-4278-a61e-11f1e81692af-service-ca\") pod \"1e6518ed-8b51-4278-a61e-11f1e81692af\" (UID: \"1e6518ed-8b51-4278-a61e-11f1e81692af\") " Jul 14 17:29:10.655803 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:10.655718 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1e6518ed-8b51-4278-a61e-11f1e81692af-oauth-serving-cert\") pod \"1e6518ed-8b51-4278-a61e-11f1e81692af\" (UID: \"1e6518ed-8b51-4278-a61e-11f1e81692af\") " Jul 14 17:29:10.656460 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:10.656423 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e6518ed-8b51-4278-a61e-11f1e81692af-console-config" (OuterVolumeSpecName: "console-config") pod "1e6518ed-8b51-4278-a61e-11f1e81692af" (UID: "1e6518ed-8b51-4278-a61e-11f1e81692af"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:29:10.656626 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:10.656434 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e6518ed-8b51-4278-a61e-11f1e81692af-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1e6518ed-8b51-4278-a61e-11f1e81692af" (UID: "1e6518ed-8b51-4278-a61e-11f1e81692af"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:29:10.656749 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:10.656540 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e6518ed-8b51-4278-a61e-11f1e81692af-service-ca" (OuterVolumeSpecName: "service-ca") pod "1e6518ed-8b51-4278-a61e-11f1e81692af" (UID: "1e6518ed-8b51-4278-a61e-11f1e81692af"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:29:10.656832 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:10.656548 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e6518ed-8b51-4278-a61e-11f1e81692af-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1e6518ed-8b51-4278-a61e-11f1e81692af" (UID: "1e6518ed-8b51-4278-a61e-11f1e81692af"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:29:10.658527 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:10.658495 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e6518ed-8b51-4278-a61e-11f1e81692af-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1e6518ed-8b51-4278-a61e-11f1e81692af" (UID: "1e6518ed-8b51-4278-a61e-11f1e81692af"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:29:10.662238 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:10.662194 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e6518ed-8b51-4278-a61e-11f1e81692af-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1e6518ed-8b51-4278-a61e-11f1e81692af" (UID: "1e6518ed-8b51-4278-a61e-11f1e81692af"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:29:10.662331 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:10.662230 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e6518ed-8b51-4278-a61e-11f1e81692af-kube-api-access-2bfd2" (OuterVolumeSpecName: "kube-api-access-2bfd2") pod "1e6518ed-8b51-4278-a61e-11f1e81692af" (UID: "1e6518ed-8b51-4278-a61e-11f1e81692af"). InnerVolumeSpecName "kube-api-access-2bfd2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:29:10.757140 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:10.757109 2554 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1e6518ed-8b51-4278-a61e-11f1e81692af-console-serving-cert\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:29:10.757140 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:10.757135 2554 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1e6518ed-8b51-4278-a61e-11f1e81692af-console-oauth-config\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:29:10.757140 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:10.757145 2554 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1e6518ed-8b51-4278-a61e-11f1e81692af-console-config\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:29:10.757378 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:10.757155 2554 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e6518ed-8b51-4278-a61e-11f1e81692af-trusted-ca-bundle\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:29:10.757378 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:10.757164 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2bfd2\" (UniqueName: \"kubernetes.io/projected/1e6518ed-8b51-4278-a61e-11f1e81692af-kube-api-access-2bfd2\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:29:10.757378 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:10.757174 2554 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1e6518ed-8b51-4278-a61e-11f1e81692af-service-ca\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:29:10.757378 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:10.757181 2554 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1e6518ed-8b51-4278-a61e-11f1e81692af-oauth-serving-cert\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:29:11.386646 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:11.386606 2554 generic.go:356] "Generic (PLEG): container finished" podID="1e6518ed-8b51-4278-a61e-11f1e81692af" containerID="c07c4997c9b7beb4f7b05f0ffd6bc57b5c7468154dd7e794ccb5e269bbee797c" exitCode=0 Jul 14 17:29:11.387038 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:11.386655 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5664c484d8-949jw" Jul 14 17:29:11.387038 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:11.386684 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5664c484d8-949jw" event={"ID":"1e6518ed-8b51-4278-a61e-11f1e81692af","Type":"ContainerDied","Data":"c07c4997c9b7beb4f7b05f0ffd6bc57b5c7468154dd7e794ccb5e269bbee797c"} Jul 14 17:29:11.387038 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:11.386735 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5664c484d8-949jw" event={"ID":"1e6518ed-8b51-4278-a61e-11f1e81692af","Type":"ContainerDied","Data":"25c56403d10d164658b5e5cbb302f06b7f00d71a7dfad9af27990425f461b829"} Jul 14 17:29:11.387038 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:11.386756 2554 scope.go:117] "RemoveContainer" containerID="c07c4997c9b7beb4f7b05f0ffd6bc57b5c7468154dd7e794ccb5e269bbee797c" Jul 14 17:29:11.395143 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:11.395125 2554 scope.go:117] "RemoveContainer" containerID="c07c4997c9b7beb4f7b05f0ffd6bc57b5c7468154dd7e794ccb5e269bbee797c" Jul 14 17:29:11.395400 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:29:11.395380 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c07c4997c9b7beb4f7b05f0ffd6bc57b5c7468154dd7e794ccb5e269bbee797c\": container with ID starting with c07c4997c9b7beb4f7b05f0ffd6bc57b5c7468154dd7e794ccb5e269bbee797c not found: ID does not exist" containerID="c07c4997c9b7beb4f7b05f0ffd6bc57b5c7468154dd7e794ccb5e269bbee797c" Jul 14 17:29:11.395476 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:11.395411 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c07c4997c9b7beb4f7b05f0ffd6bc57b5c7468154dd7e794ccb5e269bbee797c"} err="failed to get container status \"c07c4997c9b7beb4f7b05f0ffd6bc57b5c7468154dd7e794ccb5e269bbee797c\": rpc error: code = NotFound desc = could not find container \"c07c4997c9b7beb4f7b05f0ffd6bc57b5c7468154dd7e794ccb5e269bbee797c\": container with ID starting with c07c4997c9b7beb4f7b05f0ffd6bc57b5c7468154dd7e794ccb5e269bbee797c not found: ID does not exist" Jul 14 17:29:11.406922 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:11.406899 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5664c484d8-949jw"] Jul 14 17:29:11.412335 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:11.412310 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5664c484d8-949jw"] Jul 14 17:29:12.388738 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:12.388705 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e6518ed-8b51-4278-a61e-11f1e81692af" path="/var/lib/kubelet/pods/1e6518ed-8b51-4278-a61e-11f1e81692af/volumes" Jul 14 17:29:43.792715 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:43.792674 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c58849875-fbddm"] Jul 14 17:29:43.793306 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:43.793062 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e6518ed-8b51-4278-a61e-11f1e81692af" containerName="console" Jul 14 17:29:43.793306 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:43.793081 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e6518ed-8b51-4278-a61e-11f1e81692af" containerName="console" Jul 14 17:29:43.793306 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:43.793179 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e6518ed-8b51-4278-a61e-11f1e81692af" containerName="console" Jul 14 17:29:43.796427 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:43.796403 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c58849875-fbddm" Jul 14 17:29:43.805690 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:43.805662 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c58849875-fbddm"] Jul 14 17:29:43.813306 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:43.813284 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-console-serving-cert\") pod \"console-7c58849875-fbddm\" (UID: \"c77f9f28-61ed-4eb8-b9cb-205bc21d38db\") " pod="openshift-console/console-7c58849875-fbddm" Jul 14 17:29:43.813406 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:43.813319 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-console-oauth-config\") pod \"console-7c58849875-fbddm\" (UID: \"c77f9f28-61ed-4eb8-b9cb-205bc21d38db\") " pod="openshift-console/console-7c58849875-fbddm" Jul 14 17:29:43.813406 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:43.813342 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r79h6\" (UniqueName: \"kubernetes.io/projected/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-kube-api-access-r79h6\") pod \"console-7c58849875-fbddm\" (UID: \"c77f9f28-61ed-4eb8-b9cb-205bc21d38db\") " pod="openshift-console/console-7c58849875-fbddm" Jul 14 17:29:43.813406 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:43.813364 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-trusted-ca-bundle\") pod \"console-7c58849875-fbddm\" (UID: \"c77f9f28-61ed-4eb8-b9cb-205bc21d38db\") " pod="openshift-console/console-7c58849875-fbddm" Jul 14 17:29:43.813507 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:43.813407 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-oauth-serving-cert\") pod \"console-7c58849875-fbddm\" (UID: \"c77f9f28-61ed-4eb8-b9cb-205bc21d38db\") " pod="openshift-console/console-7c58849875-fbddm" Jul 14 17:29:43.813507 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:43.813477 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-service-ca\") pod \"console-7c58849875-fbddm\" (UID: \"c77f9f28-61ed-4eb8-b9cb-205bc21d38db\") " pod="openshift-console/console-7c58849875-fbddm" Jul 14 17:29:43.813569 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:43.813525 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-console-config\") pod \"console-7c58849875-fbddm\" (UID: \"c77f9f28-61ed-4eb8-b9cb-205bc21d38db\") " pod="openshift-console/console-7c58849875-fbddm" Jul 14 17:29:43.914510 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:43.914474 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-service-ca\") pod \"console-7c58849875-fbddm\" (UID: \"c77f9f28-61ed-4eb8-b9cb-205bc21d38db\") " pod="openshift-console/console-7c58849875-fbddm" Jul 14 17:29:43.914680 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:43.914519 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-console-config\") pod \"console-7c58849875-fbddm\" (UID: \"c77f9f28-61ed-4eb8-b9cb-205bc21d38db\") " pod="openshift-console/console-7c58849875-fbddm" Jul 14 17:29:43.914680 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:43.914555 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-console-serving-cert\") pod \"console-7c58849875-fbddm\" (UID: \"c77f9f28-61ed-4eb8-b9cb-205bc21d38db\") " pod="openshift-console/console-7c58849875-fbddm" Jul 14 17:29:43.914680 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:43.914581 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-console-oauth-config\") pod \"console-7c58849875-fbddm\" (UID: \"c77f9f28-61ed-4eb8-b9cb-205bc21d38db\") " pod="openshift-console/console-7c58849875-fbddm" Jul 14 17:29:43.914680 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:43.914602 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r79h6\" (UniqueName: \"kubernetes.io/projected/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-kube-api-access-r79h6\") pod \"console-7c58849875-fbddm\" (UID: \"c77f9f28-61ed-4eb8-b9cb-205bc21d38db\") " pod="openshift-console/console-7c58849875-fbddm" Jul 14 17:29:43.914680 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:43.914623 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-trusted-ca-bundle\") pod \"console-7c58849875-fbddm\" (UID: \"c77f9f28-61ed-4eb8-b9cb-205bc21d38db\") " pod="openshift-console/console-7c58849875-fbddm" Jul 14 17:29:43.914680 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:43.914669 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-oauth-serving-cert\") pod \"console-7c58849875-fbddm\" (UID: \"c77f9f28-61ed-4eb8-b9cb-205bc21d38db\") " pod="openshift-console/console-7c58849875-fbddm" Jul 14 17:29:43.915393 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:43.915365 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-service-ca\") pod \"console-7c58849875-fbddm\" (UID: \"c77f9f28-61ed-4eb8-b9cb-205bc21d38db\") " pod="openshift-console/console-7c58849875-fbddm" Jul 14 17:29:43.915393 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:43.915390 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-console-config\") pod \"console-7c58849875-fbddm\" (UID: \"c77f9f28-61ed-4eb8-b9cb-205bc21d38db\") " pod="openshift-console/console-7c58849875-fbddm" Jul 14 17:29:43.915603 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:43.915427 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-oauth-serving-cert\") pod \"console-7c58849875-fbddm\" (UID: \"c77f9f28-61ed-4eb8-b9cb-205bc21d38db\") " pod="openshift-console/console-7c58849875-fbddm" Jul 14 17:29:43.915603 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:43.915540 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-trusted-ca-bundle\") pod \"console-7c58849875-fbddm\" (UID: \"c77f9f28-61ed-4eb8-b9cb-205bc21d38db\") " pod="openshift-console/console-7c58849875-fbddm" Jul 14 17:29:43.917095 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:43.917065 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-console-serving-cert\") pod \"console-7c58849875-fbddm\" (UID: \"c77f9f28-61ed-4eb8-b9cb-205bc21d38db\") " pod="openshift-console/console-7c58849875-fbddm" Jul 14 17:29:43.917095 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:43.917090 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-console-oauth-config\") pod \"console-7c58849875-fbddm\" (UID: \"c77f9f28-61ed-4eb8-b9cb-205bc21d38db\") " pod="openshift-console/console-7c58849875-fbddm" Jul 14 17:29:43.922084 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:43.922063 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r79h6\" (UniqueName: \"kubernetes.io/projected/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-kube-api-access-r79h6\") pod \"console-7c58849875-fbddm\" (UID: \"c77f9f28-61ed-4eb8-b9cb-205bc21d38db\") " pod="openshift-console/console-7c58849875-fbddm" Jul 14 17:29:44.105118 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:44.105095 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c58849875-fbddm" Jul 14 17:29:44.231391 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:44.231328 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c58849875-fbddm"] Jul 14 17:29:44.234165 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:29:44.234139 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc77f9f28_61ed_4eb8_b9cb_205bc21d38db.slice/crio-da6eca2d7a1472f7acbd4f4c5d21e0f70d197d006d55c7d2fc68dd8088e801fa WatchSource:0}: Error finding container da6eca2d7a1472f7acbd4f4c5d21e0f70d197d006d55c7d2fc68dd8088e801fa: Status 404 returned error can't find the container with id da6eca2d7a1472f7acbd4f4c5d21e0f70d197d006d55c7d2fc68dd8088e801fa Jul 14 17:29:44.501526 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:44.501439 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c58849875-fbddm" event={"ID":"c77f9f28-61ed-4eb8-b9cb-205bc21d38db","Type":"ContainerStarted","Data":"f4aef61bbb53aacdbc3a4faa3917893e7a17f172505d75dd9ce21620c7de20fc"} Jul 14 17:29:44.501526 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:44.501496 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c58849875-fbddm" event={"ID":"c77f9f28-61ed-4eb8-b9cb-205bc21d38db","Type":"ContainerStarted","Data":"da6eca2d7a1472f7acbd4f4c5d21e0f70d197d006d55c7d2fc68dd8088e801fa"} Jul 14 17:29:44.518379 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:44.518329 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c58849875-fbddm" podStartSLOduration=1.5183143559999999 podStartE2EDuration="1.518314356s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:29:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:29:44.517949331 +0000 UTC m=+228.659198034" watchObservedRunningTime="2026-07-14 17:29:44.518314356 +0000 UTC m=+228.659563059" Jul 14 17:29:54.106055 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:54.106025 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c58849875-fbddm" Jul 14 17:29:54.106518 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:54.106078 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c58849875-fbddm" Jul 14 17:29:54.110474 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:54.110453 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c58849875-fbddm" Jul 14 17:29:54.540875 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:54.540798 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c58849875-fbddm" Jul 14 17:29:54.584637 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:29:54.584601 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c6dfccffc-dj7fd"] Jul 14 17:30:19.604747 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:19.604655 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-c6dfccffc-dj7fd" podUID="af02cb8d-c30f-4e48-9bd9-404750d96f82" containerName="console" containerID="cri-o://e330b36d775cd7b3cdf0e8d1ee8c306a2b1b22acb6af99d03acdfbe34b281e35" gracePeriod=15 Jul 14 17:30:19.848047 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:19.848021 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c6dfccffc-dj7fd" Jul 14 17:30:19.890835 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:19.890758 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af02cb8d-c30f-4e48-9bd9-404750d96f82-console-config\") pod \"af02cb8d-c30f-4e48-9bd9-404750d96f82\" (UID: \"af02cb8d-c30f-4e48-9bd9-404750d96f82\") " Jul 14 17:30:19.890835 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:19.890793 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af02cb8d-c30f-4e48-9bd9-404750d96f82-console-serving-cert\") pod \"af02cb8d-c30f-4e48-9bd9-404750d96f82\" (UID: \"af02cb8d-c30f-4e48-9bd9-404750d96f82\") " Jul 14 17:30:19.891035 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:19.890841 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af02cb8d-c30f-4e48-9bd9-404750d96f82-console-oauth-config\") pod \"af02cb8d-c30f-4e48-9bd9-404750d96f82\" (UID: \"af02cb8d-c30f-4e48-9bd9-404750d96f82\") " Jul 14 17:30:19.891035 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:19.890916 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af02cb8d-c30f-4e48-9bd9-404750d96f82-trusted-ca-bundle\") pod \"af02cb8d-c30f-4e48-9bd9-404750d96f82\" (UID: \"af02cb8d-c30f-4e48-9bd9-404750d96f82\") " Jul 14 17:30:19.891035 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:19.890950 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af02cb8d-c30f-4e48-9bd9-404750d96f82-service-ca\") pod \"af02cb8d-c30f-4e48-9bd9-404750d96f82\" (UID: \"af02cb8d-c30f-4e48-9bd9-404750d96f82\") " Jul 14 17:30:19.891035 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:19.890976 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af02cb8d-c30f-4e48-9bd9-404750d96f82-oauth-serving-cert\") pod \"af02cb8d-c30f-4e48-9bd9-404750d96f82\" (UID: \"af02cb8d-c30f-4e48-9bd9-404750d96f82\") " Jul 14 17:30:19.891035 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:19.891002 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tk6vd\" (UniqueName: \"kubernetes.io/projected/af02cb8d-c30f-4e48-9bd9-404750d96f82-kube-api-access-tk6vd\") pod \"af02cb8d-c30f-4e48-9bd9-404750d96f82\" (UID: \"af02cb8d-c30f-4e48-9bd9-404750d96f82\") " Jul 14 17:30:19.891318 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:19.891120 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af02cb8d-c30f-4e48-9bd9-404750d96f82-console-config" (OuterVolumeSpecName: "console-config") pod "af02cb8d-c30f-4e48-9bd9-404750d96f82" (UID: "af02cb8d-c30f-4e48-9bd9-404750d96f82"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:30:19.891388 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:19.891359 2554 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af02cb8d-c30f-4e48-9bd9-404750d96f82-console-config\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:30:19.891388 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:19.891372 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af02cb8d-c30f-4e48-9bd9-404750d96f82-service-ca" (OuterVolumeSpecName: "service-ca") pod "af02cb8d-c30f-4e48-9bd9-404750d96f82" (UID: "af02cb8d-c30f-4e48-9bd9-404750d96f82"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:30:19.891478 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:19.891381 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af02cb8d-c30f-4e48-9bd9-404750d96f82-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "af02cb8d-c30f-4e48-9bd9-404750d96f82" (UID: "af02cb8d-c30f-4e48-9bd9-404750d96f82"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:30:19.891478 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:19.891391 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af02cb8d-c30f-4e48-9bd9-404750d96f82-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "af02cb8d-c30f-4e48-9bd9-404750d96f82" (UID: "af02cb8d-c30f-4e48-9bd9-404750d96f82"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:30:19.893073 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:19.893049 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af02cb8d-c30f-4e48-9bd9-404750d96f82-kube-api-access-tk6vd" (OuterVolumeSpecName: "kube-api-access-tk6vd") pod "af02cb8d-c30f-4e48-9bd9-404750d96f82" (UID: "af02cb8d-c30f-4e48-9bd9-404750d96f82"). InnerVolumeSpecName "kube-api-access-tk6vd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:30:19.893472 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:19.893445 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af02cb8d-c30f-4e48-9bd9-404750d96f82-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "af02cb8d-c30f-4e48-9bd9-404750d96f82" (UID: "af02cb8d-c30f-4e48-9bd9-404750d96f82"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:30:19.893472 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:19.893463 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af02cb8d-c30f-4e48-9bd9-404750d96f82-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "af02cb8d-c30f-4e48-9bd9-404750d96f82" (UID: "af02cb8d-c30f-4e48-9bd9-404750d96f82"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:30:19.992138 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:19.992097 2554 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af02cb8d-c30f-4e48-9bd9-404750d96f82-trusted-ca-bundle\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:30:19.992138 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:19.992127 2554 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af02cb8d-c30f-4e48-9bd9-404750d96f82-service-ca\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:30:19.992138 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:19.992138 2554 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af02cb8d-c30f-4e48-9bd9-404750d96f82-oauth-serving-cert\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:30:19.992138 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:19.992146 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tk6vd\" (UniqueName: \"kubernetes.io/projected/af02cb8d-c30f-4e48-9bd9-404750d96f82-kube-api-access-tk6vd\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:30:19.992424 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:19.992157 2554 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af02cb8d-c30f-4e48-9bd9-404750d96f82-console-serving-cert\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:30:19.992424 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:19.992165 2554 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af02cb8d-c30f-4e48-9bd9-404750d96f82-console-oauth-config\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:30:20.627822 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:20.627792 2554 generic.go:356] "Generic (PLEG): container finished" podID="af02cb8d-c30f-4e48-9bd9-404750d96f82" containerID="e330b36d775cd7b3cdf0e8d1ee8c306a2b1b22acb6af99d03acdfbe34b281e35" exitCode=0 Jul 14 17:30:20.628231 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:20.627828 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c6dfccffc-dj7fd" event={"ID":"af02cb8d-c30f-4e48-9bd9-404750d96f82","Type":"ContainerDied","Data":"e330b36d775cd7b3cdf0e8d1ee8c306a2b1b22acb6af99d03acdfbe34b281e35"} Jul 14 17:30:20.628231 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:20.627839 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c6dfccffc-dj7fd" Jul 14 17:30:20.628231 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:20.627864 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c6dfccffc-dj7fd" event={"ID":"af02cb8d-c30f-4e48-9bd9-404750d96f82","Type":"ContainerDied","Data":"729fe255d48d778e5f8a46dc514f7ae5becee0a362752b96e48aac90b85a1ce5"} Jul 14 17:30:20.628231 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:20.627889 2554 scope.go:117] "RemoveContainer" containerID="e330b36d775cd7b3cdf0e8d1ee8c306a2b1b22acb6af99d03acdfbe34b281e35" Jul 14 17:30:20.635654 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:20.635629 2554 scope.go:117] "RemoveContainer" containerID="e330b36d775cd7b3cdf0e8d1ee8c306a2b1b22acb6af99d03acdfbe34b281e35" Jul 14 17:30:20.635910 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:30:20.635891 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e330b36d775cd7b3cdf0e8d1ee8c306a2b1b22acb6af99d03acdfbe34b281e35\": container with ID starting with e330b36d775cd7b3cdf0e8d1ee8c306a2b1b22acb6af99d03acdfbe34b281e35 not found: ID does not exist" containerID="e330b36d775cd7b3cdf0e8d1ee8c306a2b1b22acb6af99d03acdfbe34b281e35" Jul 14 17:30:20.635994 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:20.635918 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e330b36d775cd7b3cdf0e8d1ee8c306a2b1b22acb6af99d03acdfbe34b281e35"} err="failed to get container status \"e330b36d775cd7b3cdf0e8d1ee8c306a2b1b22acb6af99d03acdfbe34b281e35\": rpc error: code = NotFound desc = could not find container \"e330b36d775cd7b3cdf0e8d1ee8c306a2b1b22acb6af99d03acdfbe34b281e35\": container with ID starting with e330b36d775cd7b3cdf0e8d1ee8c306a2b1b22acb6af99d03acdfbe34b281e35 not found: ID does not exist" Jul 14 17:30:20.644496 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:20.644446 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c6dfccffc-dj7fd"] Jul 14 17:30:20.648779 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:20.648758 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-c6dfccffc-dj7fd"] Jul 14 17:30:22.386119 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:22.386088 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af02cb8d-c30f-4e48-9bd9-404750d96f82" path="/var/lib/kubelet/pods/af02cb8d-c30f-4e48-9bd9-404750d96f82/volumes" Jul 14 17:30:53.315810 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:53.315775 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xgqb"] Jul 14 17:30:53.316334 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:53.316084 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af02cb8d-c30f-4e48-9bd9-404750d96f82" containerName="console" Jul 14 17:30:53.316334 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:53.316096 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="af02cb8d-c30f-4e48-9bd9-404750d96f82" containerName="console" Jul 14 17:30:53.316334 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:53.316151 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="af02cb8d-c30f-4e48-9bd9-404750d96f82" containerName="console" Jul 14 17:30:53.319278 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:53.319262 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xgqb" Jul 14 17:30:53.322161 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:53.322141 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jfqff\"" Jul 14 17:30:53.322161 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:53.322153 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 14 17:30:53.323059 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:53.323039 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 14 17:30:53.328801 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:53.328779 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xgqb"] Jul 14 17:30:53.349499 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:53.349479 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2ba9e931-6909-46c2-aca1-562f64de216a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xgqb\" (UID: \"2ba9e931-6909-46c2-aca1-562f64de216a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xgqb" Jul 14 17:30:53.349586 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:53.349526 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mhnp\" (UniqueName: \"kubernetes.io/projected/2ba9e931-6909-46c2-aca1-562f64de216a-kube-api-access-7mhnp\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xgqb\" (UID: \"2ba9e931-6909-46c2-aca1-562f64de216a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xgqb" Jul 14 17:30:53.349627 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:53.349612 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2ba9e931-6909-46c2-aca1-562f64de216a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xgqb\" (UID: \"2ba9e931-6909-46c2-aca1-562f64de216a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xgqb" Jul 14 17:30:53.450307 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:53.450268 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mhnp\" (UniqueName: \"kubernetes.io/projected/2ba9e931-6909-46c2-aca1-562f64de216a-kube-api-access-7mhnp\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xgqb\" (UID: \"2ba9e931-6909-46c2-aca1-562f64de216a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xgqb" Jul 14 17:30:53.450307 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:53.450316 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2ba9e931-6909-46c2-aca1-562f64de216a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xgqb\" (UID: \"2ba9e931-6909-46c2-aca1-562f64de216a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xgqb" Jul 14 17:30:53.450446 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:53.450366 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2ba9e931-6909-46c2-aca1-562f64de216a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xgqb\" (UID: \"2ba9e931-6909-46c2-aca1-562f64de216a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xgqb" Jul 14 17:30:53.450758 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:53.450739 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2ba9e931-6909-46c2-aca1-562f64de216a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xgqb\" (UID: \"2ba9e931-6909-46c2-aca1-562f64de216a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xgqb" Jul 14 17:30:53.450758 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:53.450750 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2ba9e931-6909-46c2-aca1-562f64de216a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xgqb\" (UID: \"2ba9e931-6909-46c2-aca1-562f64de216a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xgqb" Jul 14 17:30:53.458850 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:53.458818 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mhnp\" (UniqueName: \"kubernetes.io/projected/2ba9e931-6909-46c2-aca1-562f64de216a-kube-api-access-7mhnp\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xgqb\" (UID: \"2ba9e931-6909-46c2-aca1-562f64de216a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xgqb" Jul 14 17:30:53.628295 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:53.628270 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xgqb" Jul 14 17:30:53.753835 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:53.753812 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xgqb"] Jul 14 17:30:53.756348 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:30:53.756323 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ba9e931_6909_46c2_aca1_562f64de216a.slice/crio-ff9ff0c48f7fc46b1ddd894fb1ab6face5db91d232f9bcaafc86393adaccb771 WatchSource:0}: Error finding container ff9ff0c48f7fc46b1ddd894fb1ab6face5db91d232f9bcaafc86393adaccb771: Status 404 returned error can't find the container with id ff9ff0c48f7fc46b1ddd894fb1ab6face5db91d232f9bcaafc86393adaccb771 Jul 14 17:30:54.751811 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:54.751772 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xgqb" event={"ID":"2ba9e931-6909-46c2-aca1-562f64de216a","Type":"ContainerStarted","Data":"ff9ff0c48f7fc46b1ddd894fb1ab6face5db91d232f9bcaafc86393adaccb771"} Jul 14 17:30:56.226713 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:56.226681 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pk9h2_f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726/ovn-acl-logging/0.log" Jul 14 17:30:56.230062 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:56.230038 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pk9h2_f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726/ovn-acl-logging/0.log" Jul 14 17:30:56.234473 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:30:56.234451 2554 kubelet.go:1648] "Image garbage collection succeeded" Jul 14 17:31:02.785024 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:02.784989 2554 generic.go:356] "Generic (PLEG): container finished" podID="2ba9e931-6909-46c2-aca1-562f64de216a" containerID="3fbbac89fa5f653e8c79ae36a5913f31f9ed85fec8ccaa20f6aa5e6cbb5a3e10" exitCode=0 Jul 14 17:31:02.785417 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:02.785066 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xgqb" event={"ID":"2ba9e931-6909-46c2-aca1-562f64de216a","Type":"ContainerDied","Data":"3fbbac89fa5f653e8c79ae36a5913f31f9ed85fec8ccaa20f6aa5e6cbb5a3e10"} Jul 14 17:31:02.786057 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:02.786041 2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 17:31:05.801468 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:05.801434 2554 generic.go:356] "Generic (PLEG): container finished" podID="2ba9e931-6909-46c2-aca1-562f64de216a" containerID="6a1ef43ea47dcaf7d6d217aad712adfd9f0d9b84d8281d3f423de42f7971c77f" exitCode=0 Jul 14 17:31:05.801881 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:05.801511 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xgqb" event={"ID":"2ba9e931-6909-46c2-aca1-562f64de216a","Type":"ContainerDied","Data":"6a1ef43ea47dcaf7d6d217aad712adfd9f0d9b84d8281d3f423de42f7971c77f"} Jul 14 17:31:14.839907 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:14.839873 2554 generic.go:356] "Generic (PLEG): container finished" podID="2ba9e931-6909-46c2-aca1-562f64de216a" containerID="ee09095ea85298b4959c3598a834f305c89bba5ffbadc2ed9154e13f5f51781a" exitCode=0 Jul 14 17:31:14.840345 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:14.839964 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xgqb" event={"ID":"2ba9e931-6909-46c2-aca1-562f64de216a","Type":"ContainerDied","Data":"ee09095ea85298b4959c3598a834f305c89bba5ffbadc2ed9154e13f5f51781a"} Jul 14 17:31:16.013416 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:16.013388 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xgqb" Jul 14 17:31:16.120357 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:16.120325 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2ba9e931-6909-46c2-aca1-562f64de216a-util\") pod \"2ba9e931-6909-46c2-aca1-562f64de216a\" (UID: \"2ba9e931-6909-46c2-aca1-562f64de216a\") " Jul 14 17:31:16.120530 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:16.120380 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7mhnp\" (UniqueName: \"kubernetes.io/projected/2ba9e931-6909-46c2-aca1-562f64de216a-kube-api-access-7mhnp\") pod \"2ba9e931-6909-46c2-aca1-562f64de216a\" (UID: \"2ba9e931-6909-46c2-aca1-562f64de216a\") " Jul 14 17:31:16.120530 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:16.120403 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2ba9e931-6909-46c2-aca1-562f64de216a-bundle\") pod \"2ba9e931-6909-46c2-aca1-562f64de216a\" (UID: \"2ba9e931-6909-46c2-aca1-562f64de216a\") " Jul 14 17:31:16.121008 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:16.120983 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ba9e931-6909-46c2-aca1-562f64de216a-bundle" (OuterVolumeSpecName: "bundle") pod "2ba9e931-6909-46c2-aca1-562f64de216a" (UID: "2ba9e931-6909-46c2-aca1-562f64de216a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:31:16.122525 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:16.122503 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ba9e931-6909-46c2-aca1-562f64de216a-kube-api-access-7mhnp" (OuterVolumeSpecName: "kube-api-access-7mhnp") pod "2ba9e931-6909-46c2-aca1-562f64de216a" (UID: "2ba9e931-6909-46c2-aca1-562f64de216a"). InnerVolumeSpecName "kube-api-access-7mhnp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:31:16.125195 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:16.125173 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ba9e931-6909-46c2-aca1-562f64de216a-util" (OuterVolumeSpecName: "util") pod "2ba9e931-6909-46c2-aca1-562f64de216a" (UID: "2ba9e931-6909-46c2-aca1-562f64de216a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:31:16.221173 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:16.221143 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mhnp\" (UniqueName: \"kubernetes.io/projected/2ba9e931-6909-46c2-aca1-562f64de216a-kube-api-access-7mhnp\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:31:16.221313 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:16.221187 2554 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2ba9e931-6909-46c2-aca1-562f64de216a-bundle\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:31:16.221313 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:16.221197 2554 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2ba9e931-6909-46c2-aca1-562f64de216a-util\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:31:16.848797 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:16.848716 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xgqb" event={"ID":"2ba9e931-6909-46c2-aca1-562f64de216a","Type":"ContainerDied","Data":"ff9ff0c48f7fc46b1ddd894fb1ab6face5db91d232f9bcaafc86393adaccb771"} Jul 14 17:31:16.848797 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:16.848743 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xgqb" Jul 14 17:31:16.848797 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:16.848762 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ff9ff0c48f7fc46b1ddd894fb1ab6face5db91d232f9bcaafc86393adaccb771" Jul 14 17:31:21.230270 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:21.230235 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-79h7d"] Jul 14 17:31:21.230625 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:21.230498 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ba9e931-6909-46c2-aca1-562f64de216a" containerName="pull" Jul 14 17:31:21.230625 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:21.230508 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ba9e931-6909-46c2-aca1-562f64de216a" containerName="pull" Jul 14 17:31:21.230625 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:21.230527 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ba9e931-6909-46c2-aca1-562f64de216a" containerName="extract" Jul 14 17:31:21.230625 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:21.230532 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ba9e931-6909-46c2-aca1-562f64de216a" containerName="extract" Jul 14 17:31:21.230625 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:21.230545 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ba9e931-6909-46c2-aca1-562f64de216a" containerName="util" Jul 14 17:31:21.230625 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:21.230550 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ba9e931-6909-46c2-aca1-562f64de216a" containerName="util" Jul 14 17:31:21.230625 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:21.230591 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ba9e931-6909-46c2-aca1-562f64de216a" containerName="extract" Jul 14 17:31:21.232296 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:21.232282 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-79h7d" Jul 14 17:31:21.235118 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:21.235083 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 14 17:31:21.235271 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:21.235123 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 14 17:31:21.235381 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:21.235365 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-gjqpk\"" Jul 14 17:31:21.245622 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:21.245597 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-79h7d"] Jul 14 17:31:21.357927 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:21.357896 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8kh9\" (UniqueName: \"kubernetes.io/projected/2f82ddd7-86e4-46f8-a5a1-d3fcc6141607-kube-api-access-x8kh9\") pod \"cert-manager-operator-controller-manager-55c444bf86-79h7d\" (UID: \"2f82ddd7-86e4-46f8-a5a1-d3fcc6141607\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-79h7d" Jul 14 17:31:21.358094 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:21.357941 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2f82ddd7-86e4-46f8-a5a1-d3fcc6141607-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-79h7d\" (UID: \"2f82ddd7-86e4-46f8-a5a1-d3fcc6141607\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-79h7d" Jul 14 17:31:21.459047 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:21.459011 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2f82ddd7-86e4-46f8-a5a1-d3fcc6141607-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-79h7d\" (UID: \"2f82ddd7-86e4-46f8-a5a1-d3fcc6141607\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-79h7d" Jul 14 17:31:21.459199 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:21.459079 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8kh9\" (UniqueName: \"kubernetes.io/projected/2f82ddd7-86e4-46f8-a5a1-d3fcc6141607-kube-api-access-x8kh9\") pod \"cert-manager-operator-controller-manager-55c444bf86-79h7d\" (UID: \"2f82ddd7-86e4-46f8-a5a1-d3fcc6141607\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-79h7d" Jul 14 17:31:21.459476 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:21.459456 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2f82ddd7-86e4-46f8-a5a1-d3fcc6141607-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-79h7d\" (UID: \"2f82ddd7-86e4-46f8-a5a1-d3fcc6141607\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-79h7d" Jul 14 17:31:21.467318 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:21.467294 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8kh9\" (UniqueName: \"kubernetes.io/projected/2f82ddd7-86e4-46f8-a5a1-d3fcc6141607-kube-api-access-x8kh9\") pod \"cert-manager-operator-controller-manager-55c444bf86-79h7d\" (UID: \"2f82ddd7-86e4-46f8-a5a1-d3fcc6141607\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-79h7d" Jul 14 17:31:21.541243 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:21.541131 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-79h7d" Jul 14 17:31:21.671958 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:21.671917 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-79h7d"] Jul 14 17:31:21.675163 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:31:21.675130 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f82ddd7_86e4_46f8_a5a1_d3fcc6141607.slice/crio-519719fe2a99c498cf0df7c005e1a464dd11181883b3b7ffe2ee2b2c8226b2d3 WatchSource:0}: Error finding container 519719fe2a99c498cf0df7c005e1a464dd11181883b3b7ffe2ee2b2c8226b2d3: Status 404 returned error can't find the container with id 519719fe2a99c498cf0df7c005e1a464dd11181883b3b7ffe2ee2b2c8226b2d3 Jul 14 17:31:21.867578 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:21.867549 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-79h7d" event={"ID":"2f82ddd7-86e4-46f8-a5a1-d3fcc6141607","Type":"ContainerStarted","Data":"519719fe2a99c498cf0df7c005e1a464dd11181883b3b7ffe2ee2b2c8226b2d3"} Jul 14 17:31:24.881953 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:24.881923 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-79h7d" event={"ID":"2f82ddd7-86e4-46f8-a5a1-d3fcc6141607","Type":"ContainerStarted","Data":"b6aca58b8e49e598a95188767bfdc642a025fb1e49223a5ee86b53fc826ee715"} Jul 14 17:31:24.914641 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:24.914594 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-79h7d" podStartSLOduration=1.469608223 podStartE2EDuration="3.914579179s" totalImagesPullingTime="2.444970956s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:31:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:31:24.912697735 +0000 UTC m=+329.053946439" watchObservedRunningTime="2026-07-14 17:31:24.914579179 +0000 UTC m=+329.055827883" Jul 14 17:31:25.098863 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:25.098829 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-zjbwx"] Jul 14 17:31:25.101103 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:25.101083 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-zjbwx" Jul 14 17:31:25.103629 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:25.103609 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-7hv6n\"" Jul 14 17:31:25.103714 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:25.103610 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 14 17:31:25.103714 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:25.103614 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 14 17:31:25.112410 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:25.112390 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-zjbwx"] Jul 14 17:31:25.187743 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:25.187676 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khj7s\" (UniqueName: \"kubernetes.io/projected/4bcbcfaf-f88e-47c9-aa08-73900659f123-kube-api-access-khj7s\") pod \"cert-manager-webhook-6486ffd997-zjbwx\" (UID: \"4bcbcfaf-f88e-47c9-aa08-73900659f123\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zjbwx" Jul 14 17:31:25.187851 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:25.187754 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4bcbcfaf-f88e-47c9-aa08-73900659f123-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-zjbwx\" (UID: \"4bcbcfaf-f88e-47c9-aa08-73900659f123\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zjbwx" Jul 14 17:31:25.288148 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:25.288117 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4bcbcfaf-f88e-47c9-aa08-73900659f123-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-zjbwx\" (UID: \"4bcbcfaf-f88e-47c9-aa08-73900659f123\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zjbwx" Jul 14 17:31:25.288322 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:25.288162 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khj7s\" (UniqueName: \"kubernetes.io/projected/4bcbcfaf-f88e-47c9-aa08-73900659f123-kube-api-access-khj7s\") pod \"cert-manager-webhook-6486ffd997-zjbwx\" (UID: \"4bcbcfaf-f88e-47c9-aa08-73900659f123\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zjbwx" Jul 14 17:31:25.309185 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:25.309149 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4bcbcfaf-f88e-47c9-aa08-73900659f123-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-zjbwx\" (UID: \"4bcbcfaf-f88e-47c9-aa08-73900659f123\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zjbwx" Jul 14 17:31:25.310163 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:25.310144 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khj7s\" (UniqueName: \"kubernetes.io/projected/4bcbcfaf-f88e-47c9-aa08-73900659f123-kube-api-access-khj7s\") pod \"cert-manager-webhook-6486ffd997-zjbwx\" (UID: \"4bcbcfaf-f88e-47c9-aa08-73900659f123\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zjbwx" Jul 14 17:31:25.418755 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:25.418723 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-zjbwx" Jul 14 17:31:25.552737 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:25.552705 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-zjbwx"] Jul 14 17:31:25.553620 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:31:25.553591 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4bcbcfaf_f88e_47c9_aa08_73900659f123.slice/crio-244f8fea03d0d3e748e083447b1d0a9f543fb52bf9107a0f54441b9ffa938ee7 WatchSource:0}: Error finding container 244f8fea03d0d3e748e083447b1d0a9f543fb52bf9107a0f54441b9ffa938ee7: Status 404 returned error can't find the container with id 244f8fea03d0d3e748e083447b1d0a9f543fb52bf9107a0f54441b9ffa938ee7 Jul 14 17:31:25.666229 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:25.666193 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhms77"] Jul 14 17:31:25.669046 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:25.669031 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhms77" Jul 14 17:31:25.673152 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:25.673124 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jfqff\"" Jul 14 17:31:25.673347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:25.673323 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 14 17:31:25.673674 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:25.673658 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 14 17:31:25.682118 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:25.682097 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhms77"] Jul 14 17:31:25.792675 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:25.792579 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a05b572-b9a9-47d9-acc8-ef6b0552b010-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhms77\" (UID: \"0a05b572-b9a9-47d9-acc8-ef6b0552b010\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhms77" Jul 14 17:31:25.792675 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:25.792666 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a05b572-b9a9-47d9-acc8-ef6b0552b010-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhms77\" (UID: \"0a05b572-b9a9-47d9-acc8-ef6b0552b010\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhms77" Jul 14 17:31:25.792858 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:25.792698 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zkdq\" (UniqueName: \"kubernetes.io/projected/0a05b572-b9a9-47d9-acc8-ef6b0552b010-kube-api-access-8zkdq\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhms77\" (UID: \"0a05b572-b9a9-47d9-acc8-ef6b0552b010\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhms77" Jul 14 17:31:25.887377 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:25.887342 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-zjbwx" event={"ID":"4bcbcfaf-f88e-47c9-aa08-73900659f123","Type":"ContainerStarted","Data":"244f8fea03d0d3e748e083447b1d0a9f543fb52bf9107a0f54441b9ffa938ee7"} Jul 14 17:31:25.893929 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:25.893904 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a05b572-b9a9-47d9-acc8-ef6b0552b010-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhms77\" (UID: \"0a05b572-b9a9-47d9-acc8-ef6b0552b010\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhms77" Jul 14 17:31:25.893929 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:25.893937 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zkdq\" (UniqueName: \"kubernetes.io/projected/0a05b572-b9a9-47d9-acc8-ef6b0552b010-kube-api-access-8zkdq\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhms77\" (UID: \"0a05b572-b9a9-47d9-acc8-ef6b0552b010\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhms77" Jul 14 17:31:25.894087 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:25.893966 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a05b572-b9a9-47d9-acc8-ef6b0552b010-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhms77\" (UID: \"0a05b572-b9a9-47d9-acc8-ef6b0552b010\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhms77" Jul 14 17:31:25.894316 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:25.894301 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a05b572-b9a9-47d9-acc8-ef6b0552b010-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhms77\" (UID: \"0a05b572-b9a9-47d9-acc8-ef6b0552b010\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhms77" Jul 14 17:31:25.894365 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:25.894348 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a05b572-b9a9-47d9-acc8-ef6b0552b010-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhms77\" (UID: \"0a05b572-b9a9-47d9-acc8-ef6b0552b010\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhms77" Jul 14 17:31:25.906105 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:25.906078 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zkdq\" (UniqueName: \"kubernetes.io/projected/0a05b572-b9a9-47d9-acc8-ef6b0552b010-kube-api-access-8zkdq\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhms77\" (UID: \"0a05b572-b9a9-47d9-acc8-ef6b0552b010\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhms77" Jul 14 17:31:25.979090 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:25.979059 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhms77" Jul 14 17:31:26.104300 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:26.104274 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhms77"] Jul 14 17:31:26.106390 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:31:26.106362 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a05b572_b9a9_47d9_acc8_ef6b0552b010.slice/crio-f900a2a48b56a1bfb77614945920c49d9ef10b54e0c195b39225cb493697cce5 WatchSource:0}: Error finding container f900a2a48b56a1bfb77614945920c49d9ef10b54e0c195b39225cb493697cce5: Status 404 returned error can't find the container with id f900a2a48b56a1bfb77614945920c49d9ef10b54e0c195b39225cb493697cce5 Jul 14 17:31:26.893391 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:26.893347 2554 generic.go:356] "Generic (PLEG): container finished" podID="0a05b572-b9a9-47d9-acc8-ef6b0552b010" containerID="d58da99e1224aa3b674e5ba938025a125bb9a896a501943687e6cc393b407b20" exitCode=0 Jul 14 17:31:26.893783 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:26.893445 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhms77" event={"ID":"0a05b572-b9a9-47d9-acc8-ef6b0552b010","Type":"ContainerDied","Data":"d58da99e1224aa3b674e5ba938025a125bb9a896a501943687e6cc393b407b20"} Jul 14 17:31:26.893783 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:26.893499 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhms77" event={"ID":"0a05b572-b9a9-47d9-acc8-ef6b0552b010","Type":"ContainerStarted","Data":"f900a2a48b56a1bfb77614945920c49d9ef10b54e0c195b39225cb493697cce5"} Jul 14 17:31:27.213652 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:27.213565 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-lbmxl"] Jul 14 17:31:27.215913 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:27.215891 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-lbmxl" Jul 14 17:31:27.219630 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:27.219613 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-kqb4r\"" Jul 14 17:31:27.251013 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:27.250987 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-lbmxl"] Jul 14 17:31:27.303838 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:27.303810 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q568f\" (UniqueName: \"kubernetes.io/projected/fb6b1bef-a04f-4606-913d-675f70b9cb28-kube-api-access-q568f\") pod \"cert-manager-cainjector-559cdd8c87-lbmxl\" (UID: \"fb6b1bef-a04f-4606-913d-675f70b9cb28\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-lbmxl" Jul 14 17:31:27.303973 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:27.303863 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fb6b1bef-a04f-4606-913d-675f70b9cb28-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-lbmxl\" (UID: \"fb6b1bef-a04f-4606-913d-675f70b9cb28\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-lbmxl" Jul 14 17:31:27.404671 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:27.404643 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q568f\" (UniqueName: \"kubernetes.io/projected/fb6b1bef-a04f-4606-913d-675f70b9cb28-kube-api-access-q568f\") pod \"cert-manager-cainjector-559cdd8c87-lbmxl\" (UID: \"fb6b1bef-a04f-4606-913d-675f70b9cb28\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-lbmxl" Jul 14 17:31:27.404848 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:27.404697 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fb6b1bef-a04f-4606-913d-675f70b9cb28-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-lbmxl\" (UID: \"fb6b1bef-a04f-4606-913d-675f70b9cb28\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-lbmxl" Jul 14 17:31:27.413272 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:27.413248 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q568f\" (UniqueName: \"kubernetes.io/projected/fb6b1bef-a04f-4606-913d-675f70b9cb28-kube-api-access-q568f\") pod \"cert-manager-cainjector-559cdd8c87-lbmxl\" (UID: \"fb6b1bef-a04f-4606-913d-675f70b9cb28\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-lbmxl" Jul 14 17:31:27.413370 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:27.413277 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fb6b1bef-a04f-4606-913d-675f70b9cb28-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-lbmxl\" (UID: \"fb6b1bef-a04f-4606-913d-675f70b9cb28\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-lbmxl" Jul 14 17:31:27.525075 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:27.524996 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-lbmxl" Jul 14 17:31:27.643603 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:27.643534 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-lbmxl"] Jul 14 17:31:27.646165 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:31:27.646140 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb6b1bef_a04f_4606_913d_675f70b9cb28.slice/crio-279af8e4d7f06bdd629d6cf2134c6e023f365e145f6eed40ce45f31fcba35d34 WatchSource:0}: Error finding container 279af8e4d7f06bdd629d6cf2134c6e023f365e145f6eed40ce45f31fcba35d34: Status 404 returned error can't find the container with id 279af8e4d7f06bdd629d6cf2134c6e023f365e145f6eed40ce45f31fcba35d34 Jul 14 17:31:27.900561 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:27.900525 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-lbmxl" event={"ID":"fb6b1bef-a04f-4606-913d-675f70b9cb28","Type":"ContainerStarted","Data":"279af8e4d7f06bdd629d6cf2134c6e023f365e145f6eed40ce45f31fcba35d34"} Jul 14 17:31:30.916954 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:30.916911 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-zjbwx" event={"ID":"4bcbcfaf-f88e-47c9-aa08-73900659f123","Type":"ContainerStarted","Data":"98ee0a0e1dfe8c4ff4f26b428c0e7de7e7ed40fd49599b2be96a6b31b047ccd8"} Jul 14 17:31:30.916954 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:30.916973 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-zjbwx" Jul 14 17:31:30.918743 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:30.918717 2554 generic.go:356] "Generic (PLEG): container finished" podID="0a05b572-b9a9-47d9-acc8-ef6b0552b010" containerID="92588aa0d955f70f9f7e177fa075b4fcc7c445735e53930798e431e7c57155de" exitCode=0 Jul 14 17:31:30.918870 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:30.918794 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhms77" event={"ID":"0a05b572-b9a9-47d9-acc8-ef6b0552b010","Type":"ContainerDied","Data":"92588aa0d955f70f9f7e177fa075b4fcc7c445735e53930798e431e7c57155de"} Jul 14 17:31:30.920461 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:30.920431 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-lbmxl" event={"ID":"fb6b1bef-a04f-4606-913d-675f70b9cb28","Type":"ContainerStarted","Data":"2f06247df9639b4a959f53a3f1909257702fd035a863d18b4c888219d299c9a4"} Jul 14 17:31:30.938113 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:30.934620 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-zjbwx" podStartSLOduration=0.937478942 podStartE2EDuration="5.934601598s" totalImagesPullingTime="4.997122656s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:31:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:31:30.932110264 +0000 UTC m=+335.073358963" watchObservedRunningTime="2026-07-14 17:31:30.934601598 +0000 UTC m=+335.075850303" Jul 14 17:31:30.967205 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:30.967155 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-lbmxl" podStartSLOduration=1.049574126 podStartE2EDuration="3.967134001s" totalImagesPullingTime="2.917559875s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:31:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:31:30.965204859 +0000 UTC m=+335.106453565" watchObservedRunningTime="2026-07-14 17:31:30.967134001 +0000 UTC m=+335.108382706" Jul 14 17:31:31.926483 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:31.926450 2554 generic.go:356] "Generic (PLEG): container finished" podID="0a05b572-b9a9-47d9-acc8-ef6b0552b010" containerID="2c1ff0a1662c1c36db5c2d2b7dde276b7be582d053644186622d2ffe3e8fcaa1" exitCode=0 Jul 14 17:31:31.926833 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:31.926528 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhms77" event={"ID":"0a05b572-b9a9-47d9-acc8-ef6b0552b010","Type":"ContainerDied","Data":"2c1ff0a1662c1c36db5c2d2b7dde276b7be582d053644186622d2ffe3e8fcaa1"} Jul 14 17:31:33.057413 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:33.057391 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhms77" Jul 14 17:31:33.152294 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:33.152259 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a05b572-b9a9-47d9-acc8-ef6b0552b010-util\") pod \"0a05b572-b9a9-47d9-acc8-ef6b0552b010\" (UID: \"0a05b572-b9a9-47d9-acc8-ef6b0552b010\") " Jul 14 17:31:33.152294 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:33.152311 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8zkdq\" (UniqueName: \"kubernetes.io/projected/0a05b572-b9a9-47d9-acc8-ef6b0552b010-kube-api-access-8zkdq\") pod \"0a05b572-b9a9-47d9-acc8-ef6b0552b010\" (UID: \"0a05b572-b9a9-47d9-acc8-ef6b0552b010\") " Jul 14 17:31:33.152545 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:33.152357 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a05b572-b9a9-47d9-acc8-ef6b0552b010-bundle\") pod \"0a05b572-b9a9-47d9-acc8-ef6b0552b010\" (UID: \"0a05b572-b9a9-47d9-acc8-ef6b0552b010\") " Jul 14 17:31:33.152782 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:33.152757 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a05b572-b9a9-47d9-acc8-ef6b0552b010-bundle" (OuterVolumeSpecName: "bundle") pod "0a05b572-b9a9-47d9-acc8-ef6b0552b010" (UID: "0a05b572-b9a9-47d9-acc8-ef6b0552b010"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:31:33.154390 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:33.154369 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a05b572-b9a9-47d9-acc8-ef6b0552b010-kube-api-access-8zkdq" (OuterVolumeSpecName: "kube-api-access-8zkdq") pod "0a05b572-b9a9-47d9-acc8-ef6b0552b010" (UID: "0a05b572-b9a9-47d9-acc8-ef6b0552b010"). InnerVolumeSpecName "kube-api-access-8zkdq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:31:33.158712 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:33.158689 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a05b572-b9a9-47d9-acc8-ef6b0552b010-util" (OuterVolumeSpecName: "util") pod "0a05b572-b9a9-47d9-acc8-ef6b0552b010" (UID: "0a05b572-b9a9-47d9-acc8-ef6b0552b010"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:31:33.253907 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:33.253826 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8zkdq\" (UniqueName: \"kubernetes.io/projected/0a05b572-b9a9-47d9-acc8-ef6b0552b010-kube-api-access-8zkdq\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:31:33.253907 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:33.253851 2554 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a05b572-b9a9-47d9-acc8-ef6b0552b010-bundle\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:31:33.253907 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:33.253861 2554 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a05b572-b9a9-47d9-acc8-ef6b0552b010-util\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:31:33.936740 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:33.936708 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhms77" event={"ID":"0a05b572-b9a9-47d9-acc8-ef6b0552b010","Type":"ContainerDied","Data":"f900a2a48b56a1bfb77614945920c49d9ef10b54e0c195b39225cb493697cce5"} Jul 14 17:31:33.936740 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:33.936723 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhms77" Jul 14 17:31:33.936740 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:33.936750 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f900a2a48b56a1bfb77614945920c49d9ef10b54e0c195b39225cb493697cce5" Jul 14 17:31:36.180927 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:36.180892 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-j28vm"] Jul 14 17:31:36.181310 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:36.181243 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a05b572-b9a9-47d9-acc8-ef6b0552b010" containerName="pull" Jul 14 17:31:36.181310 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:36.181258 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a05b572-b9a9-47d9-acc8-ef6b0552b010" containerName="pull" Jul 14 17:31:36.181310 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:36.181276 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a05b572-b9a9-47d9-acc8-ef6b0552b010" containerName="util" Jul 14 17:31:36.181310 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:36.181281 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a05b572-b9a9-47d9-acc8-ef6b0552b010" containerName="util" Jul 14 17:31:36.181310 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:36.181292 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a05b572-b9a9-47d9-acc8-ef6b0552b010" containerName="extract" Jul 14 17:31:36.181310 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:36.181298 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a05b572-b9a9-47d9-acc8-ef6b0552b010" containerName="extract" Jul 14 17:31:36.181513 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:36.181346 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a05b572-b9a9-47d9-acc8-ef6b0552b010" containerName="extract" Jul 14 17:31:36.185135 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:36.185120 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-j28vm" Jul 14 17:31:36.187604 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:36.187582 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-8trb6\"" Jul 14 17:31:36.191203 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:36.191183 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-j28vm"] Jul 14 17:31:36.279053 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:36.279018 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c226c575-4cd3-47c1-a347-453fec04c157-bound-sa-token\") pod \"cert-manager-55b4d954d5-j28vm\" (UID: \"c226c575-4cd3-47c1-a347-453fec04c157\") " pod="cert-manager/cert-manager-55b4d954d5-j28vm" Jul 14 17:31:36.279198 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:36.279103 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jx6br\" (UniqueName: \"kubernetes.io/projected/c226c575-4cd3-47c1-a347-453fec04c157-kube-api-access-jx6br\") pod \"cert-manager-55b4d954d5-j28vm\" (UID: \"c226c575-4cd3-47c1-a347-453fec04c157\") " pod="cert-manager/cert-manager-55b4d954d5-j28vm" Jul 14 17:31:36.379807 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:36.379783 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jx6br\" (UniqueName: \"kubernetes.io/projected/c226c575-4cd3-47c1-a347-453fec04c157-kube-api-access-jx6br\") pod \"cert-manager-55b4d954d5-j28vm\" (UID: \"c226c575-4cd3-47c1-a347-453fec04c157\") " pod="cert-manager/cert-manager-55b4d954d5-j28vm" Jul 14 17:31:36.379954 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:36.379819 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c226c575-4cd3-47c1-a347-453fec04c157-bound-sa-token\") pod \"cert-manager-55b4d954d5-j28vm\" (UID: \"c226c575-4cd3-47c1-a347-453fec04c157\") " pod="cert-manager/cert-manager-55b4d954d5-j28vm" Jul 14 17:31:36.388412 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:36.388382 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jx6br\" (UniqueName: \"kubernetes.io/projected/c226c575-4cd3-47c1-a347-453fec04c157-kube-api-access-jx6br\") pod \"cert-manager-55b4d954d5-j28vm\" (UID: \"c226c575-4cd3-47c1-a347-453fec04c157\") " pod="cert-manager/cert-manager-55b4d954d5-j28vm" Jul 14 17:31:36.388513 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:36.388380 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c226c575-4cd3-47c1-a347-453fec04c157-bound-sa-token\") pod \"cert-manager-55b4d954d5-j28vm\" (UID: \"c226c575-4cd3-47c1-a347-453fec04c157\") " pod="cert-manager/cert-manager-55b4d954d5-j28vm" Jul 14 17:31:36.494811 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:36.494730 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-j28vm" Jul 14 17:31:36.619594 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:36.619569 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-j28vm"] Jul 14 17:31:36.622007 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:31:36.621981 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc226c575_4cd3_47c1_a347_453fec04c157.slice/crio-aaea7a476d0de78f473d10a9bde0299acc9a4aa13c07810c0a95d9d1bb7615b5 WatchSource:0}: Error finding container aaea7a476d0de78f473d10a9bde0299acc9a4aa13c07810c0a95d9d1bb7615b5: Status 404 returned error can't find the container with id aaea7a476d0de78f473d10a9bde0299acc9a4aa13c07810c0a95d9d1bb7615b5 Jul 14 17:31:36.928445 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:36.928418 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-zjbwx" Jul 14 17:31:36.952612 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:36.952575 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-j28vm" event={"ID":"c226c575-4cd3-47c1-a347-453fec04c157","Type":"ContainerStarted","Data":"22d23f52c39431f15f8166885c56508312b49f37c63bb776b7b815be4c26460c"} Jul 14 17:31:36.952751 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:36.952628 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-j28vm" event={"ID":"c226c575-4cd3-47c1-a347-453fec04c157","Type":"ContainerStarted","Data":"aaea7a476d0de78f473d10a9bde0299acc9a4aa13c07810c0a95d9d1bb7615b5"} Jul 14 17:31:36.967572 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:36.967529 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-j28vm" podStartSLOduration=0.967513861 podStartE2EDuration="967.513861ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:31:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:31:36.966438696 +0000 UTC m=+341.107687409" watchObservedRunningTime="2026-07-14 17:31:36.967513861 +0000 UTC m=+341.108762564" Jul 14 17:31:43.575695 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:43.575660 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mxphq"] Jul 14 17:31:43.609233 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:43.609181 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mxphq"] Jul 14 17:31:43.609386 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:43.609294 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mxphq" Jul 14 17:31:43.612120 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:43.612098 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 14 17:31:43.612283 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:43.612165 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 14 17:31:43.612283 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:43.612207 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-s8vkc\"" Jul 14 17:31:43.743722 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:43.743687 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b05692f3-24b2-4b79-b964-7dfb9adfa5a0-tmp\") pod \"openshift-lws-operator-fd8ccff4c-mxphq\" (UID: \"b05692f3-24b2-4b79-b964-7dfb9adfa5a0\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mxphq" Jul 14 17:31:43.743889 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:43.743738 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6nnw\" (UniqueName: \"kubernetes.io/projected/b05692f3-24b2-4b79-b964-7dfb9adfa5a0-kube-api-access-k6nnw\") pod \"openshift-lws-operator-fd8ccff4c-mxphq\" (UID: \"b05692f3-24b2-4b79-b964-7dfb9adfa5a0\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mxphq" Jul 14 17:31:43.844745 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:43.844671 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6nnw\" (UniqueName: \"kubernetes.io/projected/b05692f3-24b2-4b79-b964-7dfb9adfa5a0-kube-api-access-k6nnw\") pod \"openshift-lws-operator-fd8ccff4c-mxphq\" (UID: \"b05692f3-24b2-4b79-b964-7dfb9adfa5a0\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mxphq" Jul 14 17:31:43.844865 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:43.844836 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b05692f3-24b2-4b79-b964-7dfb9adfa5a0-tmp\") pod \"openshift-lws-operator-fd8ccff4c-mxphq\" (UID: \"b05692f3-24b2-4b79-b964-7dfb9adfa5a0\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mxphq" Jul 14 17:31:43.845164 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:43.845147 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b05692f3-24b2-4b79-b964-7dfb9adfa5a0-tmp\") pod \"openshift-lws-operator-fd8ccff4c-mxphq\" (UID: \"b05692f3-24b2-4b79-b964-7dfb9adfa5a0\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mxphq" Jul 14 17:31:43.856958 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:43.856924 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6nnw\" (UniqueName: \"kubernetes.io/projected/b05692f3-24b2-4b79-b964-7dfb9adfa5a0-kube-api-access-k6nnw\") pod \"openshift-lws-operator-fd8ccff4c-mxphq\" (UID: \"b05692f3-24b2-4b79-b964-7dfb9adfa5a0\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mxphq" Jul 14 17:31:43.918347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:43.918309 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mxphq" Jul 14 17:31:44.067318 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:44.067294 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mxphq"] Jul 14 17:31:44.069323 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:31:44.069298 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb05692f3_24b2_4b79_b964_7dfb9adfa5a0.slice/crio-ccf84ee258d45d11923b6a404d18c52db14dd49a718d36731fec7425b969f565 WatchSource:0}: Error finding container ccf84ee258d45d11923b6a404d18c52db14dd49a718d36731fec7425b969f565: Status 404 returned error can't find the container with id ccf84ee258d45d11923b6a404d18c52db14dd49a718d36731fec7425b969f565 Jul 14 17:31:44.988756 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:44.988721 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mxphq" event={"ID":"b05692f3-24b2-4b79-b964-7dfb9adfa5a0","Type":"ContainerStarted","Data":"ccf84ee258d45d11923b6a404d18c52db14dd49a718d36731fec7425b969f565"} Jul 14 17:31:45.994672 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:45.994633 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mxphq" event={"ID":"b05692f3-24b2-4b79-b964-7dfb9adfa5a0","Type":"ContainerStarted","Data":"ec962e9ff068931070c277b93968e77a028b1512d3f78c399087b76b97178600"} Jul 14 17:31:46.011101 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:46.011048 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mxphq" podStartSLOduration=1.202222168 podStartE2EDuration="3.011033637s" totalImagesPullingTime="1.808811469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:31:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:31:46.009406747 +0000 UTC m=+350.150655448" watchObservedRunningTime="2026-07-14 17:31:46.011033637 +0000 UTC m=+350.152282388" Jul 14 17:31:48.608725 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:48.608648 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n75h"] Jul 14 17:31:48.611185 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:48.611168 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n75h" Jul 14 17:31:48.613409 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:48.613388 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 14 17:31:48.614523 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:48.614495 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jfqff\"" Jul 14 17:31:48.614627 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:48.614503 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 14 17:31:48.619733 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:48.619704 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n75h"] Jul 14 17:31:48.791354 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:48.791305 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67574b32-be18-49ee-b8ba-9fcd9dfb0cc8-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n75h\" (UID: \"67574b32-be18-49ee-b8ba-9fcd9dfb0cc8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n75h" Jul 14 17:31:48.791354 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:48.791356 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-276x9\" (UniqueName: \"kubernetes.io/projected/67574b32-be18-49ee-b8ba-9fcd9dfb0cc8-kube-api-access-276x9\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n75h\" (UID: \"67574b32-be18-49ee-b8ba-9fcd9dfb0cc8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n75h" Jul 14 17:31:48.791568 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:48.791386 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67574b32-be18-49ee-b8ba-9fcd9dfb0cc8-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n75h\" (UID: \"67574b32-be18-49ee-b8ba-9fcd9dfb0cc8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n75h" Jul 14 17:31:48.892740 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:48.892664 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67574b32-be18-49ee-b8ba-9fcd9dfb0cc8-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n75h\" (UID: \"67574b32-be18-49ee-b8ba-9fcd9dfb0cc8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n75h" Jul 14 17:31:48.892740 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:48.892697 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-276x9\" (UniqueName: \"kubernetes.io/projected/67574b32-be18-49ee-b8ba-9fcd9dfb0cc8-kube-api-access-276x9\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n75h\" (UID: \"67574b32-be18-49ee-b8ba-9fcd9dfb0cc8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n75h" Jul 14 17:31:48.892740 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:48.892725 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67574b32-be18-49ee-b8ba-9fcd9dfb0cc8-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n75h\" (UID: \"67574b32-be18-49ee-b8ba-9fcd9dfb0cc8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n75h" Jul 14 17:31:48.893061 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:48.893043 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67574b32-be18-49ee-b8ba-9fcd9dfb0cc8-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n75h\" (UID: \"67574b32-be18-49ee-b8ba-9fcd9dfb0cc8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n75h" Jul 14 17:31:48.893103 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:48.893086 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67574b32-be18-49ee-b8ba-9fcd9dfb0cc8-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n75h\" (UID: \"67574b32-be18-49ee-b8ba-9fcd9dfb0cc8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n75h" Jul 14 17:31:48.901081 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:48.901054 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-276x9\" (UniqueName: \"kubernetes.io/projected/67574b32-be18-49ee-b8ba-9fcd9dfb0cc8-kube-api-access-276x9\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n75h\" (UID: \"67574b32-be18-49ee-b8ba-9fcd9dfb0cc8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n75h" Jul 14 17:31:48.921046 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:48.921015 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n75h" Jul 14 17:31:49.049529 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:49.049472 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n75h"] Jul 14 17:31:49.051709 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:31:49.051682 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67574b32_be18_49ee_b8ba_9fcd9dfb0cc8.slice/crio-37869701d11e15c46f2dfabff80537061a9f353fa16bd60ad597869ebe4b25c3 WatchSource:0}: Error finding container 37869701d11e15c46f2dfabff80537061a9f353fa16bd60ad597869ebe4b25c3: Status 404 returned error can't find the container with id 37869701d11e15c46f2dfabff80537061a9f353fa16bd60ad597869ebe4b25c3 Jul 14 17:31:50.012706 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:50.012674 2554 generic.go:356] "Generic (PLEG): container finished" podID="67574b32-be18-49ee-b8ba-9fcd9dfb0cc8" containerID="fa5e33394f33a47c96ba89c4073763a1d9a63305638428b973ac0f25d48dfe84" exitCode=0 Jul 14 17:31:50.013122 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:50.012759 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n75h" event={"ID":"67574b32-be18-49ee-b8ba-9fcd9dfb0cc8","Type":"ContainerDied","Data":"fa5e33394f33a47c96ba89c4073763a1d9a63305638428b973ac0f25d48dfe84"} Jul 14 17:31:50.013122 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:50.012805 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n75h" event={"ID":"67574b32-be18-49ee-b8ba-9fcd9dfb0cc8","Type":"ContainerStarted","Data":"37869701d11e15c46f2dfabff80537061a9f353fa16bd60ad597869ebe4b25c3"} Jul 14 17:31:51.019247 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:51.019132 2554 generic.go:356] "Generic (PLEG): container finished" podID="67574b32-be18-49ee-b8ba-9fcd9dfb0cc8" containerID="77f2efe8989ad5a2ecf98134a12ef6c9f78b8ad25a3b41aae04c300227305645" exitCode=0 Jul 14 17:31:51.019247 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:51.019166 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n75h" event={"ID":"67574b32-be18-49ee-b8ba-9fcd9dfb0cc8","Type":"ContainerDied","Data":"77f2efe8989ad5a2ecf98134a12ef6c9f78b8ad25a3b41aae04c300227305645"} Jul 14 17:31:52.028445 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:52.028403 2554 generic.go:356] "Generic (PLEG): container finished" podID="67574b32-be18-49ee-b8ba-9fcd9dfb0cc8" containerID="859b8fcef35d7d1e7bc148dc8afd2aa2018f02e365efa41711dcf56933f04499" exitCode=0 Jul 14 17:31:52.028811 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:52.028492 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n75h" event={"ID":"67574b32-be18-49ee-b8ba-9fcd9dfb0cc8","Type":"ContainerDied","Data":"859b8fcef35d7d1e7bc148dc8afd2aa2018f02e365efa41711dcf56933f04499"} Jul 14 17:31:53.158405 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:53.158384 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n75h" Jul 14 17:31:53.328068 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:53.327974 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67574b32-be18-49ee-b8ba-9fcd9dfb0cc8-util\") pod \"67574b32-be18-49ee-b8ba-9fcd9dfb0cc8\" (UID: \"67574b32-be18-49ee-b8ba-9fcd9dfb0cc8\") " Jul 14 17:31:53.328068 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:53.328025 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-276x9\" (UniqueName: \"kubernetes.io/projected/67574b32-be18-49ee-b8ba-9fcd9dfb0cc8-kube-api-access-276x9\") pod \"67574b32-be18-49ee-b8ba-9fcd9dfb0cc8\" (UID: \"67574b32-be18-49ee-b8ba-9fcd9dfb0cc8\") " Jul 14 17:31:53.328068 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:53.328049 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67574b32-be18-49ee-b8ba-9fcd9dfb0cc8-bundle\") pod \"67574b32-be18-49ee-b8ba-9fcd9dfb0cc8\" (UID: \"67574b32-be18-49ee-b8ba-9fcd9dfb0cc8\") " Jul 14 17:31:53.328795 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:53.328764 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67574b32-be18-49ee-b8ba-9fcd9dfb0cc8-bundle" (OuterVolumeSpecName: "bundle") pod "67574b32-be18-49ee-b8ba-9fcd9dfb0cc8" (UID: "67574b32-be18-49ee-b8ba-9fcd9dfb0cc8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:31:53.330087 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:53.330066 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67574b32-be18-49ee-b8ba-9fcd9dfb0cc8-kube-api-access-276x9" (OuterVolumeSpecName: "kube-api-access-276x9") pod "67574b32-be18-49ee-b8ba-9fcd9dfb0cc8" (UID: "67574b32-be18-49ee-b8ba-9fcd9dfb0cc8"). InnerVolumeSpecName "kube-api-access-276x9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:31:53.333959 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:53.333927 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67574b32-be18-49ee-b8ba-9fcd9dfb0cc8-util" (OuterVolumeSpecName: "util") pod "67574b32-be18-49ee-b8ba-9fcd9dfb0cc8" (UID: "67574b32-be18-49ee-b8ba-9fcd9dfb0cc8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:31:53.429315 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:53.429286 2554 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67574b32-be18-49ee-b8ba-9fcd9dfb0cc8-util\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:31:53.429315 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:53.429307 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-276x9\" (UniqueName: \"kubernetes.io/projected/67574b32-be18-49ee-b8ba-9fcd9dfb0cc8-kube-api-access-276x9\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:31:53.429315 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:53.429319 2554 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67574b32-be18-49ee-b8ba-9fcd9dfb0cc8-bundle\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:31:54.038819 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:54.038788 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n75h" event={"ID":"67574b32-be18-49ee-b8ba-9fcd9dfb0cc8","Type":"ContainerDied","Data":"37869701d11e15c46f2dfabff80537061a9f353fa16bd60ad597869ebe4b25c3"} Jul 14 17:31:54.038981 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:54.038834 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="37869701d11e15c46f2dfabff80537061a9f353fa16bd60ad597869ebe4b25c3" Jul 14 17:31:54.038981 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:31:54.038800 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n75h" Jul 14 17:32:01.451258 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:01.451224 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7cb94d9664-482xx"] Jul 14 17:32:01.451714 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:01.451513 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67574b32-be18-49ee-b8ba-9fcd9dfb0cc8" containerName="util" Jul 14 17:32:01.451714 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:01.451524 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="67574b32-be18-49ee-b8ba-9fcd9dfb0cc8" containerName="util" Jul 14 17:32:01.451714 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:01.451538 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67574b32-be18-49ee-b8ba-9fcd9dfb0cc8" containerName="pull" Jul 14 17:32:01.451714 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:01.451545 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="67574b32-be18-49ee-b8ba-9fcd9dfb0cc8" containerName="pull" Jul 14 17:32:01.451714 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:01.451561 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67574b32-be18-49ee-b8ba-9fcd9dfb0cc8" containerName="extract" Jul 14 17:32:01.451714 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:01.451566 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="67574b32-be18-49ee-b8ba-9fcd9dfb0cc8" containerName="extract" Jul 14 17:32:01.451714 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:01.451610 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="67574b32-be18-49ee-b8ba-9fcd9dfb0cc8" containerName="extract" Jul 14 17:32:01.453985 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:01.453964 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7cb94d9664-482xx" Jul 14 17:32:01.456832 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:01.456812 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 14 17:32:01.458175 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:01.458157 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 14 17:32:01.458295 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:01.458186 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 14 17:32:01.458295 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:01.458188 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-8xbhw\"" Jul 14 17:32:01.466234 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:01.466201 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7cb94d9664-482xx"] Jul 14 17:32:01.491133 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:01.491110 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/11bc76d0-ed17-4b7f-b7e3-b8d2458b136e-metrics-cert\") pod \"lws-controller-manager-7cb94d9664-482xx\" (UID: \"11bc76d0-ed17-4b7f-b7e3-b8d2458b136e\") " pod="openshift-lws-operator/lws-controller-manager-7cb94d9664-482xx" Jul 14 17:32:01.491133 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:01.491139 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/11bc76d0-ed17-4b7f-b7e3-b8d2458b136e-manager-config\") pod \"lws-controller-manager-7cb94d9664-482xx\" (UID: \"11bc76d0-ed17-4b7f-b7e3-b8d2458b136e\") " pod="openshift-lws-operator/lws-controller-manager-7cb94d9664-482xx" Jul 14 17:32:01.491324 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:01.491164 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/11bc76d0-ed17-4b7f-b7e3-b8d2458b136e-cert\") pod \"lws-controller-manager-7cb94d9664-482xx\" (UID: \"11bc76d0-ed17-4b7f-b7e3-b8d2458b136e\") " pod="openshift-lws-operator/lws-controller-manager-7cb94d9664-482xx" Jul 14 17:32:01.491324 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:01.491180 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54kf8\" (UniqueName: \"kubernetes.io/projected/11bc76d0-ed17-4b7f-b7e3-b8d2458b136e-kube-api-access-54kf8\") pod \"lws-controller-manager-7cb94d9664-482xx\" (UID: \"11bc76d0-ed17-4b7f-b7e3-b8d2458b136e\") " pod="openshift-lws-operator/lws-controller-manager-7cb94d9664-482xx" Jul 14 17:32:01.591586 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:01.591553 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/11bc76d0-ed17-4b7f-b7e3-b8d2458b136e-metrics-cert\") pod \"lws-controller-manager-7cb94d9664-482xx\" (UID: \"11bc76d0-ed17-4b7f-b7e3-b8d2458b136e\") " pod="openshift-lws-operator/lws-controller-manager-7cb94d9664-482xx" Jul 14 17:32:01.591586 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:01.591587 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/11bc76d0-ed17-4b7f-b7e3-b8d2458b136e-manager-config\") pod \"lws-controller-manager-7cb94d9664-482xx\" (UID: \"11bc76d0-ed17-4b7f-b7e3-b8d2458b136e\") " pod="openshift-lws-operator/lws-controller-manager-7cb94d9664-482xx" Jul 14 17:32:01.591793 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:01.591642 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/11bc76d0-ed17-4b7f-b7e3-b8d2458b136e-cert\") pod \"lws-controller-manager-7cb94d9664-482xx\" (UID: \"11bc76d0-ed17-4b7f-b7e3-b8d2458b136e\") " pod="openshift-lws-operator/lws-controller-manager-7cb94d9664-482xx" Jul 14 17:32:01.591793 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:01.591664 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54kf8\" (UniqueName: \"kubernetes.io/projected/11bc76d0-ed17-4b7f-b7e3-b8d2458b136e-kube-api-access-54kf8\") pod \"lws-controller-manager-7cb94d9664-482xx\" (UID: \"11bc76d0-ed17-4b7f-b7e3-b8d2458b136e\") " pod="openshift-lws-operator/lws-controller-manager-7cb94d9664-482xx" Jul 14 17:32:01.592395 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:01.592372 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/11bc76d0-ed17-4b7f-b7e3-b8d2458b136e-manager-config\") pod \"lws-controller-manager-7cb94d9664-482xx\" (UID: \"11bc76d0-ed17-4b7f-b7e3-b8d2458b136e\") " pod="openshift-lws-operator/lws-controller-manager-7cb94d9664-482xx" Jul 14 17:32:01.594046 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:01.594018 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/11bc76d0-ed17-4b7f-b7e3-b8d2458b136e-metrics-cert\") pod \"lws-controller-manager-7cb94d9664-482xx\" (UID: \"11bc76d0-ed17-4b7f-b7e3-b8d2458b136e\") " pod="openshift-lws-operator/lws-controller-manager-7cb94d9664-482xx" Jul 14 17:32:01.594134 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:01.594069 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/11bc76d0-ed17-4b7f-b7e3-b8d2458b136e-cert\") pod \"lws-controller-manager-7cb94d9664-482xx\" (UID: \"11bc76d0-ed17-4b7f-b7e3-b8d2458b136e\") " pod="openshift-lws-operator/lws-controller-manager-7cb94d9664-482xx" Jul 14 17:32:01.613818 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:01.613795 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54kf8\" (UniqueName: \"kubernetes.io/projected/11bc76d0-ed17-4b7f-b7e3-b8d2458b136e-kube-api-access-54kf8\") pod \"lws-controller-manager-7cb94d9664-482xx\" (UID: \"11bc76d0-ed17-4b7f-b7e3-b8d2458b136e\") " pod="openshift-lws-operator/lws-controller-manager-7cb94d9664-482xx" Jul 14 17:32:01.762787 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:01.762708 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7cb94d9664-482xx" Jul 14 17:32:01.901777 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:01.901754 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7cb94d9664-482xx"] Jul 14 17:32:01.903761 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:32:01.903737 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11bc76d0_ed17_4b7f_b7e3_b8d2458b136e.slice/crio-4af13f563df77bc0bbc2b5f82e8b93607f0a42c08432f02344d3a13c6a923b19 WatchSource:0}: Error finding container 4af13f563df77bc0bbc2b5f82e8b93607f0a42c08432f02344d3a13c6a923b19: Status 404 returned error can't find the container with id 4af13f563df77bc0bbc2b5f82e8b93607f0a42c08432f02344d3a13c6a923b19 Jul 14 17:32:02.074603 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:02.074512 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7cb94d9664-482xx" event={"ID":"11bc76d0-ed17-4b7f-b7e3-b8d2458b136e","Type":"ContainerStarted","Data":"4af13f563df77bc0bbc2b5f82e8b93607f0a42c08432f02344d3a13c6a923b19"} Jul 14 17:32:03.870194 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:03.870158 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f54d68b84-sl8jk"] Jul 14 17:32:03.872746 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:03.872724 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7f54d68b84-sl8jk" Jul 14 17:32:03.876039 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:03.876009 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 14 17:32:03.876240 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:03.876018 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 14 17:32:03.876365 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:03.876342 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 14 17:32:03.876528 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:03.876352 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-4xg4v\"" Jul 14 17:32:03.877551 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:03.877322 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 14 17:32:03.890030 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:03.889978 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f54d68b84-sl8jk"] Jul 14 17:32:03.908237 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:03.908201 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b4e50eb9-ba03-4bcd-82fe-3bf657fab044-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f54d68b84-sl8jk\" (UID: \"b4e50eb9-ba03-4bcd-82fe-3bf657fab044\") " pod="opendatahub/opendatahub-operator-controller-manager-7f54d68b84-sl8jk" Jul 14 17:32:03.908346 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:03.908277 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b4e50eb9-ba03-4bcd-82fe-3bf657fab044-tmp\") pod \"opendatahub-operator-controller-manager-7f54d68b84-sl8jk\" (UID: \"b4e50eb9-ba03-4bcd-82fe-3bf657fab044\") " pod="opendatahub/opendatahub-operator-controller-manager-7f54d68b84-sl8jk" Jul 14 17:32:03.908346 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:03.908326 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b4e50eb9-ba03-4bcd-82fe-3bf657fab044-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f54d68b84-sl8jk\" (UID: \"b4e50eb9-ba03-4bcd-82fe-3bf657fab044\") " pod="opendatahub/opendatahub-operator-controller-manager-7f54d68b84-sl8jk" Jul 14 17:32:03.908431 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:03.908400 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b4e50eb9-ba03-4bcd-82fe-3bf657fab044-manifests\") pod \"opendatahub-operator-controller-manager-7f54d68b84-sl8jk\" (UID: \"b4e50eb9-ba03-4bcd-82fe-3bf657fab044\") " pod="opendatahub/opendatahub-operator-controller-manager-7f54d68b84-sl8jk" Jul 14 17:32:03.908431 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:03.908425 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptg7t\" (UniqueName: \"kubernetes.io/projected/b4e50eb9-ba03-4bcd-82fe-3bf657fab044-kube-api-access-ptg7t\") pod \"opendatahub-operator-controller-manager-7f54d68b84-sl8jk\" (UID: \"b4e50eb9-ba03-4bcd-82fe-3bf657fab044\") " pod="opendatahub/opendatahub-operator-controller-manager-7f54d68b84-sl8jk" Jul 14 17:32:04.008760 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:04.008730 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b4e50eb9-ba03-4bcd-82fe-3bf657fab044-manifests\") pod \"opendatahub-operator-controller-manager-7f54d68b84-sl8jk\" (UID: \"b4e50eb9-ba03-4bcd-82fe-3bf657fab044\") " pod="opendatahub/opendatahub-operator-controller-manager-7f54d68b84-sl8jk" Jul 14 17:32:04.008907 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:04.008764 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptg7t\" (UniqueName: \"kubernetes.io/projected/b4e50eb9-ba03-4bcd-82fe-3bf657fab044-kube-api-access-ptg7t\") pod \"opendatahub-operator-controller-manager-7f54d68b84-sl8jk\" (UID: \"b4e50eb9-ba03-4bcd-82fe-3bf657fab044\") " pod="opendatahub/opendatahub-operator-controller-manager-7f54d68b84-sl8jk" Jul 14 17:32:04.008907 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:04.008790 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b4e50eb9-ba03-4bcd-82fe-3bf657fab044-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f54d68b84-sl8jk\" (UID: \"b4e50eb9-ba03-4bcd-82fe-3bf657fab044\") " pod="opendatahub/opendatahub-operator-controller-manager-7f54d68b84-sl8jk" Jul 14 17:32:04.009018 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:04.008921 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b4e50eb9-ba03-4bcd-82fe-3bf657fab044-tmp\") pod \"opendatahub-operator-controller-manager-7f54d68b84-sl8jk\" (UID: \"b4e50eb9-ba03-4bcd-82fe-3bf657fab044\") " pod="opendatahub/opendatahub-operator-controller-manager-7f54d68b84-sl8jk" Jul 14 17:32:04.009018 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:04.008982 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b4e50eb9-ba03-4bcd-82fe-3bf657fab044-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f54d68b84-sl8jk\" (UID: \"b4e50eb9-ba03-4bcd-82fe-3bf657fab044\") " pod="opendatahub/opendatahub-operator-controller-manager-7f54d68b84-sl8jk" Jul 14 17:32:04.009182 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:04.009107 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b4e50eb9-ba03-4bcd-82fe-3bf657fab044-manifests\") pod \"opendatahub-operator-controller-manager-7f54d68b84-sl8jk\" (UID: \"b4e50eb9-ba03-4bcd-82fe-3bf657fab044\") " pod="opendatahub/opendatahub-operator-controller-manager-7f54d68b84-sl8jk" Jul 14 17:32:04.010984 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:04.010959 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b4e50eb9-ba03-4bcd-82fe-3bf657fab044-tmp\") pod \"opendatahub-operator-controller-manager-7f54d68b84-sl8jk\" (UID: \"b4e50eb9-ba03-4bcd-82fe-3bf657fab044\") " pod="opendatahub/opendatahub-operator-controller-manager-7f54d68b84-sl8jk" Jul 14 17:32:04.011193 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:04.011176 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b4e50eb9-ba03-4bcd-82fe-3bf657fab044-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f54d68b84-sl8jk\" (UID: \"b4e50eb9-ba03-4bcd-82fe-3bf657fab044\") " pod="opendatahub/opendatahub-operator-controller-manager-7f54d68b84-sl8jk" Jul 14 17:32:04.011265 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:04.011243 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b4e50eb9-ba03-4bcd-82fe-3bf657fab044-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f54d68b84-sl8jk\" (UID: \"b4e50eb9-ba03-4bcd-82fe-3bf657fab044\") " pod="opendatahub/opendatahub-operator-controller-manager-7f54d68b84-sl8jk" Jul 14 17:32:04.022010 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:04.021987 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptg7t\" (UniqueName: \"kubernetes.io/projected/b4e50eb9-ba03-4bcd-82fe-3bf657fab044-kube-api-access-ptg7t\") pod \"opendatahub-operator-controller-manager-7f54d68b84-sl8jk\" (UID: \"b4e50eb9-ba03-4bcd-82fe-3bf657fab044\") " pod="opendatahub/opendatahub-operator-controller-manager-7f54d68b84-sl8jk" Jul 14 17:32:04.085749 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:04.085726 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7cb94d9664-482xx" event={"ID":"11bc76d0-ed17-4b7f-b7e3-b8d2458b136e","Type":"ContainerStarted","Data":"cd36f9bed2e5272e7f6ce3cee97cab43c50f20b7af0e510f14a20cf5228bfe11"} Jul 14 17:32:04.085859 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:04.085840 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7cb94d9664-482xx" Jul 14 17:32:04.103486 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:04.103447 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7cb94d9664-482xx" podStartSLOduration=1.375018789 podStartE2EDuration="3.103434286s" totalImagesPullingTime="1.728415497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:32:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:32:04.102567406 +0000 UTC m=+368.243816123" watchObservedRunningTime="2026-07-14 17:32:04.103434286 +0000 UTC m=+368.244682989" Jul 14 17:32:04.185266 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:04.185177 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7f54d68b84-sl8jk" Jul 14 17:32:04.347077 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:04.347045 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f54d68b84-sl8jk"] Jul 14 17:32:04.349834 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:32:04.349804 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4e50eb9_ba03_4bcd_82fe_3bf657fab044.slice/crio-556375199dbc6da7284e079294391cd5763e4009633161a333197cd0fb86fb8b WatchSource:0}: Error finding container 556375199dbc6da7284e079294391cd5763e4009633161a333197cd0fb86fb8b: Status 404 returned error can't find the container with id 556375199dbc6da7284e079294391cd5763e4009633161a333197cd0fb86fb8b Jul 14 17:32:05.092685 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:05.092642 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f54d68b84-sl8jk" event={"ID":"b4e50eb9-ba03-4bcd-82fe-3bf657fab044","Type":"ContainerStarted","Data":"556375199dbc6da7284e079294391cd5763e4009633161a333197cd0fb86fb8b"} Jul 14 17:32:08.110153 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:08.110122 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f54d68b84-sl8jk" event={"ID":"b4e50eb9-ba03-4bcd-82fe-3bf657fab044","Type":"ContainerStarted","Data":"2edea51811cce81e78e905c356d9b8cd4f64ac9ae5b6e800ffc4f4355af1cfc8"} Jul 14 17:32:09.115524 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:09.115488 2554 generic.go:356] "Generic (PLEG): container finished" podID="b4e50eb9-ba03-4bcd-82fe-3bf657fab044" containerID="2edea51811cce81e78e905c356d9b8cd4f64ac9ae5b6e800ffc4f4355af1cfc8" exitCode=0 Jul 14 17:32:09.115884 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:09.115560 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f54d68b84-sl8jk" event={"ID":"b4e50eb9-ba03-4bcd-82fe-3bf657fab044","Type":"ContainerDied","Data":"2edea51811cce81e78e905c356d9b8cd4f64ac9ae5b6e800ffc4f4355af1cfc8"} Jul 14 17:32:10.121283 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:10.121252 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f54d68b84-sl8jk" event={"ID":"b4e50eb9-ba03-4bcd-82fe-3bf657fab044","Type":"ContainerStarted","Data":"67a31900b8a08e026034bcc96f6907347c618ed2cc6083f1969ce37cf0e57ea4"} Jul 14 17:32:10.121616 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:10.121385 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7f54d68b84-sl8jk" Jul 14 17:32:10.143733 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:10.143693 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7f54d68b84-sl8jk" podStartSLOduration=2.564823766 podStartE2EDuration="7.143679178s" totalImagesPullingTime="3.511230553s" totalInitContainerRuntime="1.067624859s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:32:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:32:10.141626654 +0000 UTC m=+374.282875360" watchObservedRunningTime="2026-07-14 17:32:10.143679178 +0000 UTC m=+374.284927953" Jul 14 17:32:15.094928 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:15.094895 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7cb94d9664-482xx" Jul 14 17:32:21.127261 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:21.127207 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7f54d68b84-sl8jk" Jul 14 17:32:24.963468 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:24.963439 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-87kp7"] Jul 14 17:32:24.967089 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:24.967064 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-87kp7" Jul 14 17:32:24.969893 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:24.969873 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 14 17:32:24.980122 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:24.980100 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-87kp7"] Jul 14 17:32:25.076130 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:25.076097 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9htj\" (UniqueName: \"kubernetes.io/projected/392297d1-a5b9-4967-9cd6-8d019503f41a-kube-api-access-g9htj\") pod \"model-serving-api-54d89bd5fc-87kp7\" (UID: \"392297d1-a5b9-4967-9cd6-8d019503f41a\") " pod="opendatahub/model-serving-api-54d89bd5fc-87kp7" Jul 14 17:32:25.076330 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:25.076158 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/392297d1-a5b9-4967-9cd6-8d019503f41a-tls-certs\") pod \"model-serving-api-54d89bd5fc-87kp7\" (UID: \"392297d1-a5b9-4967-9cd6-8d019503f41a\") " pod="opendatahub/model-serving-api-54d89bd5fc-87kp7" Jul 14 17:32:25.177530 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:25.177495 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/392297d1-a5b9-4967-9cd6-8d019503f41a-tls-certs\") pod \"model-serving-api-54d89bd5fc-87kp7\" (UID: \"392297d1-a5b9-4967-9cd6-8d019503f41a\") " pod="opendatahub/model-serving-api-54d89bd5fc-87kp7" Jul 14 17:32:25.177721 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:25.177578 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9htj\" (UniqueName: \"kubernetes.io/projected/392297d1-a5b9-4967-9cd6-8d019503f41a-kube-api-access-g9htj\") pod \"model-serving-api-54d89bd5fc-87kp7\" (UID: \"392297d1-a5b9-4967-9cd6-8d019503f41a\") " pod="opendatahub/model-serving-api-54d89bd5fc-87kp7" Jul 14 17:32:25.177721 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:32:25.177657 2554 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 14 17:32:25.177951 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:32:25.177722 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/392297d1-a5b9-4967-9cd6-8d019503f41a-tls-certs podName:392297d1-a5b9-4967-9cd6-8d019503f41a nodeName:}" failed. No retries permitted until 2026-07-14 17:32:25.677703735 +0000 UTC m=+389.818952438 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/392297d1-a5b9-4967-9cd6-8d019503f41a-tls-certs") pod "model-serving-api-54d89bd5fc-87kp7" (UID: "392297d1-a5b9-4967-9cd6-8d019503f41a") : secret "model-serving-api-tls" not found Jul 14 17:32:25.196119 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:25.196093 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9htj\" (UniqueName: \"kubernetes.io/projected/392297d1-a5b9-4967-9cd6-8d019503f41a-kube-api-access-g9htj\") pod \"model-serving-api-54d89bd5fc-87kp7\" (UID: \"392297d1-a5b9-4967-9cd6-8d019503f41a\") " pod="opendatahub/model-serving-api-54d89bd5fc-87kp7" Jul 14 17:32:25.681791 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:25.681757 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/392297d1-a5b9-4967-9cd6-8d019503f41a-tls-certs\") pod \"model-serving-api-54d89bd5fc-87kp7\" (UID: \"392297d1-a5b9-4967-9cd6-8d019503f41a\") " pod="opendatahub/model-serving-api-54d89bd5fc-87kp7" Jul 14 17:32:25.684159 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:25.684136 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/392297d1-a5b9-4967-9cd6-8d019503f41a-tls-certs\") pod \"model-serving-api-54d89bd5fc-87kp7\" (UID: \"392297d1-a5b9-4967-9cd6-8d019503f41a\") " pod="opendatahub/model-serving-api-54d89bd5fc-87kp7" Jul 14 17:32:25.877069 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:25.877041 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-87kp7" Jul 14 17:32:26.001981 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:26.001958 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-87kp7"] Jul 14 17:32:26.003848 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:32:26.003826 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod392297d1_a5b9_4967_9cd6_8d019503f41a.slice/crio-a03e370f7a2930f85868cbf672016d425a792373c8e8348fc6ad0bc04a244f5e WatchSource:0}: Error finding container a03e370f7a2930f85868cbf672016d425a792373c8e8348fc6ad0bc04a244f5e: Status 404 returned error can't find the container with id a03e370f7a2930f85868cbf672016d425a792373c8e8348fc6ad0bc04a244f5e Jul 14 17:32:26.192732 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:26.192695 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-87kp7" event={"ID":"392297d1-a5b9-4967-9cd6-8d019503f41a","Type":"ContainerStarted","Data":"a03e370f7a2930f85868cbf672016d425a792373c8e8348fc6ad0bc04a244f5e"} Jul 14 17:32:28.203877 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:28.203844 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-87kp7" event={"ID":"392297d1-a5b9-4967-9cd6-8d019503f41a","Type":"ContainerStarted","Data":"7560470e1d154045076e1232e8eb835717a906611d677e1d650bb6dbdf542a17"} Jul 14 17:32:28.204349 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:28.203965 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-87kp7" Jul 14 17:32:28.222613 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:28.222571 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-87kp7" podStartSLOduration=2.13043137 podStartE2EDuration="4.22255682s" totalImagesPullingTime="2.09212545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:32:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:32:28.22091811 +0000 UTC m=+392.362166817" watchObservedRunningTime="2026-07-14 17:32:28.22255682 +0000 UTC m=+392.363805523" Jul 14 17:32:31.782623 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:31.782592 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p5nfg"] Jul 14 17:32:31.785937 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:31.785915 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p5nfg" Jul 14 17:32:31.789389 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:31.789367 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 14 17:32:31.789492 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:31.789370 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 14 17:32:31.789660 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:31.789643 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 14 17:32:31.790150 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:31.790135 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 14 17:32:31.790225 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:31.790146 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-zq6vk\"" Jul 14 17:32:31.790578 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:31.790564 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 14 17:32:31.793439 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:31.793423 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 14 17:32:31.803708 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:31.803690 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p5nfg"] Jul 14 17:32:31.829902 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:31.829878 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5b775922-7477-4d67-91c5-d359eda0ebae-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-p5nfg\" (UID: \"5b775922-7477-4d67-91c5-d359eda0ebae\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p5nfg" Jul 14 17:32:31.829997 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:31.829905 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjvgr\" (UniqueName: \"kubernetes.io/projected/5b775922-7477-4d67-91c5-d359eda0ebae-kube-api-access-zjvgr\") pod \"istiod-openshift-gateway-94bb8fbfd-p5nfg\" (UID: \"5b775922-7477-4d67-91c5-d359eda0ebae\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p5nfg" Jul 14 17:32:31.829997 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:31.829944 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5b775922-7477-4d67-91c5-d359eda0ebae-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-p5nfg\" (UID: \"5b775922-7477-4d67-91c5-d359eda0ebae\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p5nfg" Jul 14 17:32:31.830104 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:31.830007 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b775922-7477-4d67-91c5-d359eda0ebae-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-p5nfg\" (UID: \"5b775922-7477-4d67-91c5-d359eda0ebae\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p5nfg" Jul 14 17:32:31.830104 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:31.830046 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5b775922-7477-4d67-91c5-d359eda0ebae-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-p5nfg\" (UID: \"5b775922-7477-4d67-91c5-d359eda0ebae\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p5nfg" Jul 14 17:32:31.830233 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:31.830125 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b775922-7477-4d67-91c5-d359eda0ebae-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-p5nfg\" (UID: \"5b775922-7477-4d67-91c5-d359eda0ebae\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p5nfg" Jul 14 17:32:31.830233 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:31.830182 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5b775922-7477-4d67-91c5-d359eda0ebae-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-p5nfg\" (UID: \"5b775922-7477-4d67-91c5-d359eda0ebae\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p5nfg" Jul 14 17:32:31.931493 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:31.931470 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b775922-7477-4d67-91c5-d359eda0ebae-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-p5nfg\" (UID: \"5b775922-7477-4d67-91c5-d359eda0ebae\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p5nfg" Jul 14 17:32:31.931619 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:31.931499 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5b775922-7477-4d67-91c5-d359eda0ebae-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-p5nfg\" (UID: \"5b775922-7477-4d67-91c5-d359eda0ebae\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p5nfg" Jul 14 17:32:31.931619 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:31.931531 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5b775922-7477-4d67-91c5-d359eda0ebae-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-p5nfg\" (UID: \"5b775922-7477-4d67-91c5-d359eda0ebae\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p5nfg" Jul 14 17:32:31.931619 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:31.931555 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjvgr\" (UniqueName: \"kubernetes.io/projected/5b775922-7477-4d67-91c5-d359eda0ebae-kube-api-access-zjvgr\") pod \"istiod-openshift-gateway-94bb8fbfd-p5nfg\" (UID: \"5b775922-7477-4d67-91c5-d359eda0ebae\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p5nfg" Jul 14 17:32:31.931619 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:31.931594 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5b775922-7477-4d67-91c5-d359eda0ebae-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-p5nfg\" (UID: \"5b775922-7477-4d67-91c5-d359eda0ebae\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p5nfg" Jul 14 17:32:31.931877 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:31.931625 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b775922-7477-4d67-91c5-d359eda0ebae-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-p5nfg\" (UID: \"5b775922-7477-4d67-91c5-d359eda0ebae\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p5nfg" Jul 14 17:32:31.931877 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:31.931643 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5b775922-7477-4d67-91c5-d359eda0ebae-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-p5nfg\" (UID: \"5b775922-7477-4d67-91c5-d359eda0ebae\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p5nfg" Jul 14 17:32:31.932313 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:31.932285 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5b775922-7477-4d67-91c5-d359eda0ebae-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-p5nfg\" (UID: \"5b775922-7477-4d67-91c5-d359eda0ebae\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p5nfg" Jul 14 17:32:31.933794 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:31.933765 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b775922-7477-4d67-91c5-d359eda0ebae-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-p5nfg\" (UID: \"5b775922-7477-4d67-91c5-d359eda0ebae\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p5nfg" Jul 14 17:32:31.933963 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:31.933943 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5b775922-7477-4d67-91c5-d359eda0ebae-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-p5nfg\" (UID: \"5b775922-7477-4d67-91c5-d359eda0ebae\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p5nfg" Jul 14 17:32:31.934025 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:31.933964 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5b775922-7477-4d67-91c5-d359eda0ebae-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-p5nfg\" (UID: \"5b775922-7477-4d67-91c5-d359eda0ebae\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p5nfg" Jul 14 17:32:31.934025 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:31.933970 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5b775922-7477-4d67-91c5-d359eda0ebae-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-p5nfg\" (UID: \"5b775922-7477-4d67-91c5-d359eda0ebae\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p5nfg" Jul 14 17:32:31.940371 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:31.940346 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b775922-7477-4d67-91c5-d359eda0ebae-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-p5nfg\" (UID: \"5b775922-7477-4d67-91c5-d359eda0ebae\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p5nfg" Jul 14 17:32:31.940687 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:31.940670 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjvgr\" (UniqueName: \"kubernetes.io/projected/5b775922-7477-4d67-91c5-d359eda0ebae-kube-api-access-zjvgr\") pod \"istiod-openshift-gateway-94bb8fbfd-p5nfg\" (UID: \"5b775922-7477-4d67-91c5-d359eda0ebae\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p5nfg" Jul 14 17:32:32.095311 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:32.095207 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p5nfg" Jul 14 17:32:32.223474 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:32.223449 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p5nfg"] Jul 14 17:32:32.225224 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:32:32.225190 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b775922_7477_4d67_91c5_d359eda0ebae.slice/crio-551c283fbdec9a02245ccbc730ee8dc8f91b4bf4627fc5e646b8474a6c85fa99 WatchSource:0}: Error finding container 551c283fbdec9a02245ccbc730ee8dc8f91b4bf4627fc5e646b8474a6c85fa99: Status 404 returned error can't find the container with id 551c283fbdec9a02245ccbc730ee8dc8f91b4bf4627fc5e646b8474a6c85fa99 Jul 14 17:32:33.228108 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:33.228053 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p5nfg" event={"ID":"5b775922-7477-4d67-91c5-d359eda0ebae","Type":"ContainerStarted","Data":"551c283fbdec9a02245ccbc730ee8dc8f91b4bf4627fc5e646b8474a6c85fa99"} Jul 14 17:32:34.389896 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:34.389847 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 17:32:34.390164 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:34.389936 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 17:32:35.239543 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:35.239452 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p5nfg" event={"ID":"5b775922-7477-4d67-91c5-d359eda0ebae","Type":"ContainerStarted","Data":"edc89905bcce736fb4db8ba6e23b95d76349dd60e9702e40f43897f64e914617"} Jul 14 17:32:35.239716 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:35.239608 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p5nfg" Jul 14 17:32:35.241135 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:35.241114 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p5nfg" Jul 14 17:32:35.264741 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:35.264701 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p5nfg" podStartSLOduration=2.102346268 podStartE2EDuration="4.264689024s" totalImagesPullingTime="2.162342756s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:32:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:32:35.263578197 +0000 UTC m=+399.404826901" watchObservedRunningTime="2026-07-14 17:32:35.264689024 +0000 UTC m=+399.405937727" Jul 14 17:32:39.212457 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:32:39.212425 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-87kp7" Jul 14 17:33:39.025750 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:39.025713 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5b6ff48bc7-dss8d"] Jul 14 17:33:39.029000 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:39.028982 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b6ff48bc7-dss8d" Jul 14 17:33:39.039140 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:39.039108 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b6ff48bc7-dss8d"] Jul 14 17:33:39.087793 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:39.087757 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7c9401b9-de05-45f2-84fa-882ffbee6e53-oauth-serving-cert\") pod \"console-5b6ff48bc7-dss8d\" (UID: \"7c9401b9-de05-45f2-84fa-882ffbee6e53\") " pod="openshift-console/console-5b6ff48bc7-dss8d" Jul 14 17:33:39.087966 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:39.087809 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7c9401b9-de05-45f2-84fa-882ffbee6e53-console-oauth-config\") pod \"console-5b6ff48bc7-dss8d\" (UID: \"7c9401b9-de05-45f2-84fa-882ffbee6e53\") " pod="openshift-console/console-5b6ff48bc7-dss8d" Jul 14 17:33:39.087966 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:39.087857 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7c9401b9-de05-45f2-84fa-882ffbee6e53-console-config\") pod \"console-5b6ff48bc7-dss8d\" (UID: \"7c9401b9-de05-45f2-84fa-882ffbee6e53\") " pod="openshift-console/console-5b6ff48bc7-dss8d" Jul 14 17:33:39.087966 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:39.087954 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7c9401b9-de05-45f2-84fa-882ffbee6e53-console-serving-cert\") pod \"console-5b6ff48bc7-dss8d\" (UID: \"7c9401b9-de05-45f2-84fa-882ffbee6e53\") " pod="openshift-console/console-5b6ff48bc7-dss8d" Jul 14 17:33:39.088087 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:39.088002 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c9401b9-de05-45f2-84fa-882ffbee6e53-trusted-ca-bundle\") pod \"console-5b6ff48bc7-dss8d\" (UID: \"7c9401b9-de05-45f2-84fa-882ffbee6e53\") " pod="openshift-console/console-5b6ff48bc7-dss8d" Jul 14 17:33:39.088087 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:39.088019 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7c9401b9-de05-45f2-84fa-882ffbee6e53-service-ca\") pod \"console-5b6ff48bc7-dss8d\" (UID: \"7c9401b9-de05-45f2-84fa-882ffbee6e53\") " pod="openshift-console/console-5b6ff48bc7-dss8d" Jul 14 17:33:39.088087 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:39.088039 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lm276\" (UniqueName: \"kubernetes.io/projected/7c9401b9-de05-45f2-84fa-882ffbee6e53-kube-api-access-lm276\") pod \"console-5b6ff48bc7-dss8d\" (UID: \"7c9401b9-de05-45f2-84fa-882ffbee6e53\") " pod="openshift-console/console-5b6ff48bc7-dss8d" Jul 14 17:33:39.188461 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:39.188425 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7c9401b9-de05-45f2-84fa-882ffbee6e53-console-serving-cert\") pod \"console-5b6ff48bc7-dss8d\" (UID: \"7c9401b9-de05-45f2-84fa-882ffbee6e53\") " pod="openshift-console/console-5b6ff48bc7-dss8d" Jul 14 17:33:39.188669 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:39.188474 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c9401b9-de05-45f2-84fa-882ffbee6e53-trusted-ca-bundle\") pod \"console-5b6ff48bc7-dss8d\" (UID: \"7c9401b9-de05-45f2-84fa-882ffbee6e53\") " pod="openshift-console/console-5b6ff48bc7-dss8d" Jul 14 17:33:39.188669 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:39.188490 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7c9401b9-de05-45f2-84fa-882ffbee6e53-service-ca\") pod \"console-5b6ff48bc7-dss8d\" (UID: \"7c9401b9-de05-45f2-84fa-882ffbee6e53\") " pod="openshift-console/console-5b6ff48bc7-dss8d" Jul 14 17:33:39.188669 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:39.188508 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lm276\" (UniqueName: \"kubernetes.io/projected/7c9401b9-de05-45f2-84fa-882ffbee6e53-kube-api-access-lm276\") pod \"console-5b6ff48bc7-dss8d\" (UID: \"7c9401b9-de05-45f2-84fa-882ffbee6e53\") " pod="openshift-console/console-5b6ff48bc7-dss8d" Jul 14 17:33:39.188669 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:39.188565 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7c9401b9-de05-45f2-84fa-882ffbee6e53-oauth-serving-cert\") pod \"console-5b6ff48bc7-dss8d\" (UID: \"7c9401b9-de05-45f2-84fa-882ffbee6e53\") " pod="openshift-console/console-5b6ff48bc7-dss8d" Jul 14 17:33:39.188669 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:39.188609 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7c9401b9-de05-45f2-84fa-882ffbee6e53-console-oauth-config\") pod \"console-5b6ff48bc7-dss8d\" (UID: \"7c9401b9-de05-45f2-84fa-882ffbee6e53\") " pod="openshift-console/console-5b6ff48bc7-dss8d" Jul 14 17:33:39.188669 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:39.188635 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7c9401b9-de05-45f2-84fa-882ffbee6e53-console-config\") pod \"console-5b6ff48bc7-dss8d\" (UID: \"7c9401b9-de05-45f2-84fa-882ffbee6e53\") " pod="openshift-console/console-5b6ff48bc7-dss8d" Jul 14 17:33:39.189325 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:39.189294 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7c9401b9-de05-45f2-84fa-882ffbee6e53-service-ca\") pod \"console-5b6ff48bc7-dss8d\" (UID: \"7c9401b9-de05-45f2-84fa-882ffbee6e53\") " pod="openshift-console/console-5b6ff48bc7-dss8d" Jul 14 17:33:39.189450 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:39.189385 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7c9401b9-de05-45f2-84fa-882ffbee6e53-oauth-serving-cert\") pod \"console-5b6ff48bc7-dss8d\" (UID: \"7c9401b9-de05-45f2-84fa-882ffbee6e53\") " pod="openshift-console/console-5b6ff48bc7-dss8d" Jul 14 17:33:39.189492 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:39.189469 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c9401b9-de05-45f2-84fa-882ffbee6e53-trusted-ca-bundle\") pod \"console-5b6ff48bc7-dss8d\" (UID: \"7c9401b9-de05-45f2-84fa-882ffbee6e53\") " pod="openshift-console/console-5b6ff48bc7-dss8d" Jul 14 17:33:39.189549 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:39.189526 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7c9401b9-de05-45f2-84fa-882ffbee6e53-console-config\") pod \"console-5b6ff48bc7-dss8d\" (UID: \"7c9401b9-de05-45f2-84fa-882ffbee6e53\") " pod="openshift-console/console-5b6ff48bc7-dss8d" Jul 14 17:33:39.191110 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:39.191080 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7c9401b9-de05-45f2-84fa-882ffbee6e53-console-oauth-config\") pod \"console-5b6ff48bc7-dss8d\" (UID: \"7c9401b9-de05-45f2-84fa-882ffbee6e53\") " pod="openshift-console/console-5b6ff48bc7-dss8d" Jul 14 17:33:39.191201 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:39.191110 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7c9401b9-de05-45f2-84fa-882ffbee6e53-console-serving-cert\") pod \"console-5b6ff48bc7-dss8d\" (UID: \"7c9401b9-de05-45f2-84fa-882ffbee6e53\") " pod="openshift-console/console-5b6ff48bc7-dss8d" Jul 14 17:33:39.200115 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:39.200091 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lm276\" (UniqueName: \"kubernetes.io/projected/7c9401b9-de05-45f2-84fa-882ffbee6e53-kube-api-access-lm276\") pod \"console-5b6ff48bc7-dss8d\" (UID: \"7c9401b9-de05-45f2-84fa-882ffbee6e53\") " pod="openshift-console/console-5b6ff48bc7-dss8d" Jul 14 17:33:39.338718 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:39.338631 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b6ff48bc7-dss8d" Jul 14 17:33:39.469188 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:39.469110 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b6ff48bc7-dss8d"] Jul 14 17:33:39.471383 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:33:39.471357 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c9401b9_de05_45f2_84fa_882ffbee6e53.slice/crio-feb3a14f36659114ce717cda68c59aa47dad2c73d5a700f4c129d1a792700ff0 WatchSource:0}: Error finding container feb3a14f36659114ce717cda68c59aa47dad2c73d5a700f4c129d1a792700ff0: Status 404 returned error can't find the container with id feb3a14f36659114ce717cda68c59aa47dad2c73d5a700f4c129d1a792700ff0 Jul 14 17:33:39.538622 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:39.538587 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b6ff48bc7-dss8d" event={"ID":"7c9401b9-de05-45f2-84fa-882ffbee6e53","Type":"ContainerStarted","Data":"feb3a14f36659114ce717cda68c59aa47dad2c73d5a700f4c129d1a792700ff0"} Jul 14 17:33:40.179937 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:40.179883 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wcnzk"] Jul 14 17:33:40.184432 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:40.184406 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wcnzk" Jul 14 17:33:40.187137 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:40.187114 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 14 17:33:40.187280 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:40.187194 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-wbl4l\"" Jul 14 17:33:40.188379 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:40.188350 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 14 17:33:40.191155 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:40.191127 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wcnzk"] Jul 14 17:33:40.297944 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:40.297899 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0045d523-1b12-4f86-bc45-e490ec45a739-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wcnzk\" (UID: \"0045d523-1b12-4f86-bc45-e490ec45a739\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wcnzk" Jul 14 17:33:40.297944 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:40.297944 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0045d523-1b12-4f86-bc45-e490ec45a739-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wcnzk\" (UID: \"0045d523-1b12-4f86-bc45-e490ec45a739\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wcnzk" Jul 14 17:33:40.298149 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:40.298050 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzkj9\" (UniqueName: \"kubernetes.io/projected/0045d523-1b12-4f86-bc45-e490ec45a739-kube-api-access-tzkj9\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wcnzk\" (UID: \"0045d523-1b12-4f86-bc45-e490ec45a739\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wcnzk" Jul 14 17:33:40.399385 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:40.399351 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzkj9\" (UniqueName: \"kubernetes.io/projected/0045d523-1b12-4f86-bc45-e490ec45a739-kube-api-access-tzkj9\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wcnzk\" (UID: \"0045d523-1b12-4f86-bc45-e490ec45a739\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wcnzk" Jul 14 17:33:40.399633 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:40.399418 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0045d523-1b12-4f86-bc45-e490ec45a739-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wcnzk\" (UID: \"0045d523-1b12-4f86-bc45-e490ec45a739\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wcnzk" Jul 14 17:33:40.399633 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:40.399438 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0045d523-1b12-4f86-bc45-e490ec45a739-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wcnzk\" (UID: \"0045d523-1b12-4f86-bc45-e490ec45a739\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wcnzk" Jul 14 17:33:40.399765 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:40.399750 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0045d523-1b12-4f86-bc45-e490ec45a739-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wcnzk\" (UID: \"0045d523-1b12-4f86-bc45-e490ec45a739\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wcnzk" Jul 14 17:33:40.399818 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:40.399800 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0045d523-1b12-4f86-bc45-e490ec45a739-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wcnzk\" (UID: \"0045d523-1b12-4f86-bc45-e490ec45a739\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wcnzk" Jul 14 17:33:40.407321 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:40.407295 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzkj9\" (UniqueName: \"kubernetes.io/projected/0045d523-1b12-4f86-bc45-e490ec45a739-kube-api-access-tzkj9\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wcnzk\" (UID: \"0045d523-1b12-4f86-bc45-e490ec45a739\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wcnzk" Jul 14 17:33:40.503515 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:40.503438 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wcnzk" Jul 14 17:33:40.547724 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:40.547685 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b6ff48bc7-dss8d" event={"ID":"7c9401b9-de05-45f2-84fa-882ffbee6e53","Type":"ContainerStarted","Data":"646c95f81aec4e55778a92d2a45830cf209544b9b1bac7a739aaabc07c2d3dd4"} Jul 14 17:33:40.564959 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:40.564890 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5b6ff48bc7-dss8d" podStartSLOduration=1.564873375 podStartE2EDuration="1.564873375s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:33:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:33:40.564226748 +0000 UTC m=+464.705475449" watchObservedRunningTime="2026-07-14 17:33:40.564873375 +0000 UTC m=+464.706122079" Jul 14 17:33:40.631625 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:40.631601 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wcnzk"] Jul 14 17:33:40.633436 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:33:40.633411 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0045d523_1b12_4f86_bc45_e490ec45a739.slice/crio-34f9dc1951a2dd520df7ff077c85a20969091753f14b88b3cfdb40371a2aaf69 WatchSource:0}: Error finding container 34f9dc1951a2dd520df7ff077c85a20969091753f14b88b3cfdb40371a2aaf69: Status 404 returned error can't find the container with id 34f9dc1951a2dd520df7ff077c85a20969091753f14b88b3cfdb40371a2aaf69 Jul 14 17:33:40.983540 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:40.983505 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2rwx"] Jul 14 17:33:40.986909 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:40.986893 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2rwx" Jul 14 17:33:40.995306 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:40.995276 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2rwx"] Jul 14 17:33:41.106748 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:41.106719 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4e9fc54e-180e-44ec-aaa1-76638d1f30ab-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2rwx\" (UID: \"4e9fc54e-180e-44ec-aaa1-76638d1f30ab\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2rwx" Jul 14 17:33:41.106748 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:41.106756 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzd8l\" (UniqueName: \"kubernetes.io/projected/4e9fc54e-180e-44ec-aaa1-76638d1f30ab-kube-api-access-jzd8l\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2rwx\" (UID: \"4e9fc54e-180e-44ec-aaa1-76638d1f30ab\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2rwx" Jul 14 17:33:41.106966 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:41.106848 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4e9fc54e-180e-44ec-aaa1-76638d1f30ab-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2rwx\" (UID: \"4e9fc54e-180e-44ec-aaa1-76638d1f30ab\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2rwx" Jul 14 17:33:41.208298 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:41.208262 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4e9fc54e-180e-44ec-aaa1-76638d1f30ab-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2rwx\" (UID: \"4e9fc54e-180e-44ec-aaa1-76638d1f30ab\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2rwx" Jul 14 17:33:41.208298 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:41.208311 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4e9fc54e-180e-44ec-aaa1-76638d1f30ab-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2rwx\" (UID: \"4e9fc54e-180e-44ec-aaa1-76638d1f30ab\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2rwx" Jul 14 17:33:41.208796 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:41.208333 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzd8l\" (UniqueName: \"kubernetes.io/projected/4e9fc54e-180e-44ec-aaa1-76638d1f30ab-kube-api-access-jzd8l\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2rwx\" (UID: \"4e9fc54e-180e-44ec-aaa1-76638d1f30ab\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2rwx" Jul 14 17:33:41.208796 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:41.208667 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4e9fc54e-180e-44ec-aaa1-76638d1f30ab-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2rwx\" (UID: \"4e9fc54e-180e-44ec-aaa1-76638d1f30ab\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2rwx" Jul 14 17:33:41.208796 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:41.208716 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4e9fc54e-180e-44ec-aaa1-76638d1f30ab-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2rwx\" (UID: \"4e9fc54e-180e-44ec-aaa1-76638d1f30ab\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2rwx" Jul 14 17:33:41.217040 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:41.217014 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzd8l\" (UniqueName: \"kubernetes.io/projected/4e9fc54e-180e-44ec-aaa1-76638d1f30ab-kube-api-access-jzd8l\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2rwx\" (UID: \"4e9fc54e-180e-44ec-aaa1-76638d1f30ab\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2rwx" Jul 14 17:33:41.296278 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:41.296159 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2rwx" Jul 14 17:33:41.422588 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:41.422563 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2rwx"] Jul 14 17:33:41.424195 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:33:41.424163 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e9fc54e_180e_44ec_aaa1_76638d1f30ab.slice/crio-53859df757313cf4e3056d9eda9f3c3dd0997e370df5df96c735a8ef8b214941 WatchSource:0}: Error finding container 53859df757313cf4e3056d9eda9f3c3dd0997e370df5df96c735a8ef8b214941: Status 404 returned error can't find the container with id 53859df757313cf4e3056d9eda9f3c3dd0997e370df5df96c735a8ef8b214941 Jul 14 17:33:41.555063 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:41.554982 2554 generic.go:356] "Generic (PLEG): container finished" podID="4e9fc54e-180e-44ec-aaa1-76638d1f30ab" containerID="e1486aee3b5be17372073e9a263bc60b98c59c2900dcd68a70659d9a6dd0cbfb" exitCode=0 Jul 14 17:33:41.555233 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:41.555079 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2rwx" event={"ID":"4e9fc54e-180e-44ec-aaa1-76638d1f30ab","Type":"ContainerDied","Data":"e1486aee3b5be17372073e9a263bc60b98c59c2900dcd68a70659d9a6dd0cbfb"} Jul 14 17:33:41.555233 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:41.555140 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2rwx" event={"ID":"4e9fc54e-180e-44ec-aaa1-76638d1f30ab","Type":"ContainerStarted","Data":"53859df757313cf4e3056d9eda9f3c3dd0997e370df5df96c735a8ef8b214941"} Jul 14 17:33:41.556720 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:41.556701 2554 generic.go:356] "Generic (PLEG): container finished" podID="0045d523-1b12-4f86-bc45-e490ec45a739" containerID="0bdd7cef84ff30e0b61758be3b89913d23d45192c2680745b444fc9b22483034" exitCode=0 Jul 14 17:33:41.556816 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:41.556749 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wcnzk" event={"ID":"0045d523-1b12-4f86-bc45-e490ec45a739","Type":"ContainerDied","Data":"0bdd7cef84ff30e0b61758be3b89913d23d45192c2680745b444fc9b22483034"} Jul 14 17:33:41.556816 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:41.556802 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wcnzk" event={"ID":"0045d523-1b12-4f86-bc45-e490ec45a739","Type":"ContainerStarted","Data":"34f9dc1951a2dd520df7ff077c85a20969091753f14b88b3cfdb40371a2aaf69"} Jul 14 17:33:41.581226 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:41.581193 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09znmf"] Jul 14 17:33:41.584749 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:41.584732 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09znmf" Jul 14 17:33:41.594331 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:41.594288 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09znmf"] Jul 14 17:33:41.713619 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:41.713584 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1bf1435a-7689-4366-879b-562c985f0d65-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09znmf\" (UID: \"1bf1435a-7689-4366-879b-562c985f0d65\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09znmf" Jul 14 17:33:41.713786 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:41.713634 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1bf1435a-7689-4366-879b-562c985f0d65-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09znmf\" (UID: \"1bf1435a-7689-4366-879b-562c985f0d65\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09znmf" Jul 14 17:33:41.713786 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:41.713719 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgxxg\" (UniqueName: \"kubernetes.io/projected/1bf1435a-7689-4366-879b-562c985f0d65-kube-api-access-qgxxg\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09znmf\" (UID: \"1bf1435a-7689-4366-879b-562c985f0d65\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09znmf" Jul 14 17:33:41.814926 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:41.814824 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1bf1435a-7689-4366-879b-562c985f0d65-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09znmf\" (UID: \"1bf1435a-7689-4366-879b-562c985f0d65\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09znmf" Jul 14 17:33:41.814926 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:41.814888 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgxxg\" (UniqueName: \"kubernetes.io/projected/1bf1435a-7689-4366-879b-562c985f0d65-kube-api-access-qgxxg\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09znmf\" (UID: \"1bf1435a-7689-4366-879b-562c985f0d65\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09znmf" Jul 14 17:33:41.814926 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:41.814937 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1bf1435a-7689-4366-879b-562c985f0d65-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09znmf\" (UID: \"1bf1435a-7689-4366-879b-562c985f0d65\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09znmf" Jul 14 17:33:41.815326 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:41.815304 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1bf1435a-7689-4366-879b-562c985f0d65-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09znmf\" (UID: \"1bf1435a-7689-4366-879b-562c985f0d65\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09znmf" Jul 14 17:33:41.815326 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:41.815317 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1bf1435a-7689-4366-879b-562c985f0d65-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09znmf\" (UID: \"1bf1435a-7689-4366-879b-562c985f0d65\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09znmf" Jul 14 17:33:41.823962 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:41.823939 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgxxg\" (UniqueName: \"kubernetes.io/projected/1bf1435a-7689-4366-879b-562c985f0d65-kube-api-access-qgxxg\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09znmf\" (UID: \"1bf1435a-7689-4366-879b-562c985f0d65\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09znmf" Jul 14 17:33:41.895949 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:41.895919 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09znmf" Jul 14 17:33:41.982348 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:41.982310 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mvnqq"] Jul 14 17:33:41.986023 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:41.986004 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mvnqq" Jul 14 17:33:41.994581 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:41.994524 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mvnqq"] Jul 14 17:33:42.038669 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:42.037051 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09znmf"] Jul 14 17:33:42.040629 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:33:42.040598 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bf1435a_7689_4366_879b_562c985f0d65.slice/crio-77e759050f582198df1e20d57860f45461159433388e5f99b251a7351e42d74d WatchSource:0}: Error finding container 77e759050f582198df1e20d57860f45461159433388e5f99b251a7351e42d74d: Status 404 returned error can't find the container with id 77e759050f582198df1e20d57860f45461159433388e5f99b251a7351e42d74d Jul 14 17:33:42.117963 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:42.117932 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sktdk\" (UniqueName: \"kubernetes.io/projected/1f45d5d7-31ab-42d5-a787-c06a1fe43b2f-kube-api-access-sktdk\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mvnqq\" (UID: \"1f45d5d7-31ab-42d5-a787-c06a1fe43b2f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mvnqq" Jul 14 17:33:42.118122 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:42.117972 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1f45d5d7-31ab-42d5-a787-c06a1fe43b2f-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mvnqq\" (UID: \"1f45d5d7-31ab-42d5-a787-c06a1fe43b2f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mvnqq" Jul 14 17:33:42.118122 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:42.117995 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1f45d5d7-31ab-42d5-a787-c06a1fe43b2f-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mvnqq\" (UID: \"1f45d5d7-31ab-42d5-a787-c06a1fe43b2f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mvnqq" Jul 14 17:33:42.218769 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:42.218736 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1f45d5d7-31ab-42d5-a787-c06a1fe43b2f-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mvnqq\" (UID: \"1f45d5d7-31ab-42d5-a787-c06a1fe43b2f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mvnqq" Jul 14 17:33:42.219109 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:42.218775 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1f45d5d7-31ab-42d5-a787-c06a1fe43b2f-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mvnqq\" (UID: \"1f45d5d7-31ab-42d5-a787-c06a1fe43b2f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mvnqq" Jul 14 17:33:42.219109 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:42.218851 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sktdk\" (UniqueName: \"kubernetes.io/projected/1f45d5d7-31ab-42d5-a787-c06a1fe43b2f-kube-api-access-sktdk\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mvnqq\" (UID: \"1f45d5d7-31ab-42d5-a787-c06a1fe43b2f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mvnqq" Jul 14 17:33:42.219109 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:42.219061 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1f45d5d7-31ab-42d5-a787-c06a1fe43b2f-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mvnqq\" (UID: \"1f45d5d7-31ab-42d5-a787-c06a1fe43b2f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mvnqq" Jul 14 17:33:42.219254 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:42.219237 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1f45d5d7-31ab-42d5-a787-c06a1fe43b2f-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mvnqq\" (UID: \"1f45d5d7-31ab-42d5-a787-c06a1fe43b2f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mvnqq" Jul 14 17:33:42.229124 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:42.229100 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sktdk\" (UniqueName: \"kubernetes.io/projected/1f45d5d7-31ab-42d5-a787-c06a1fe43b2f-kube-api-access-sktdk\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mvnqq\" (UID: \"1f45d5d7-31ab-42d5-a787-c06a1fe43b2f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mvnqq" Jul 14 17:33:42.297003 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:42.296967 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mvnqq" Jul 14 17:33:42.567893 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:42.567844 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2rwx" event={"ID":"4e9fc54e-180e-44ec-aaa1-76638d1f30ab","Type":"ContainerStarted","Data":"e8726c8520cbf1300d750c99d40321b691b761c08f831c363c0a5347bb14db8b"} Jul 14 17:33:42.568631 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:42.568607 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mvnqq"] Jul 14 17:33:42.570589 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:42.570394 2554 generic.go:356] "Generic (PLEG): container finished" podID="1bf1435a-7689-4366-879b-562c985f0d65" containerID="6663a8812cc444c849d3c1d401f8c5436c337c52f5fe63a8902544967597ef29" exitCode=0 Jul 14 17:33:42.570589 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:42.570464 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09znmf" event={"ID":"1bf1435a-7689-4366-879b-562c985f0d65","Type":"ContainerDied","Data":"6663a8812cc444c849d3c1d401f8c5436c337c52f5fe63a8902544967597ef29"} Jul 14 17:33:42.570589 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:42.570520 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09znmf" event={"ID":"1bf1435a-7689-4366-879b-562c985f0d65","Type":"ContainerStarted","Data":"77e759050f582198df1e20d57860f45461159433388e5f99b251a7351e42d74d"} Jul 14 17:33:42.572002 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:33:42.571980 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f45d5d7_31ab_42d5_a787_c06a1fe43b2f.slice/crio-885980252fdac403d015bcb16623917234efdc7c068c19963c20ae2b55a41d11 WatchSource:0}: Error finding container 885980252fdac403d015bcb16623917234efdc7c068c19963c20ae2b55a41d11: Status 404 returned error can't find the container with id 885980252fdac403d015bcb16623917234efdc7c068c19963c20ae2b55a41d11 Jul 14 17:33:42.573586 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:42.573556 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wcnzk" event={"ID":"0045d523-1b12-4f86-bc45-e490ec45a739","Type":"ContainerStarted","Data":"825e1f94077e388f6619466f987a818cdbd0391b7cb68c2900da391aebb39a0e"} Jul 14 17:33:43.579666 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:43.579639 2554 generic.go:356] "Generic (PLEG): container finished" podID="1f45d5d7-31ab-42d5-a787-c06a1fe43b2f" containerID="cbe8d1a5043cb274f6f51d406d0d873d56ba500e7e9c4e10f5f7be7dcc972385" exitCode=0 Jul 14 17:33:43.579986 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:43.579717 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mvnqq" event={"ID":"1f45d5d7-31ab-42d5-a787-c06a1fe43b2f","Type":"ContainerDied","Data":"cbe8d1a5043cb274f6f51d406d0d873d56ba500e7e9c4e10f5f7be7dcc972385"} Jul 14 17:33:43.579986 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:43.579768 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mvnqq" event={"ID":"1f45d5d7-31ab-42d5-a787-c06a1fe43b2f","Type":"ContainerStarted","Data":"885980252fdac403d015bcb16623917234efdc7c068c19963c20ae2b55a41d11"} Jul 14 17:33:43.581591 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:43.581569 2554 generic.go:356] "Generic (PLEG): container finished" podID="0045d523-1b12-4f86-bc45-e490ec45a739" containerID="825e1f94077e388f6619466f987a818cdbd0391b7cb68c2900da391aebb39a0e" exitCode=0 Jul 14 17:33:43.581692 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:43.581665 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wcnzk" event={"ID":"0045d523-1b12-4f86-bc45-e490ec45a739","Type":"ContainerDied","Data":"825e1f94077e388f6619466f987a818cdbd0391b7cb68c2900da391aebb39a0e"} Jul 14 17:33:43.583560 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:43.583533 2554 generic.go:356] "Generic (PLEG): container finished" podID="4e9fc54e-180e-44ec-aaa1-76638d1f30ab" containerID="e8726c8520cbf1300d750c99d40321b691b761c08f831c363c0a5347bb14db8b" exitCode=0 Jul 14 17:33:43.583656 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:43.583594 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2rwx" event={"ID":"4e9fc54e-180e-44ec-aaa1-76638d1f30ab","Type":"ContainerDied","Data":"e8726c8520cbf1300d750c99d40321b691b761c08f831c363c0a5347bb14db8b"} Jul 14 17:33:43.585577 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:43.585556 2554 generic.go:356] "Generic (PLEG): container finished" podID="1bf1435a-7689-4366-879b-562c985f0d65" containerID="0a00e025a31be4e137266fb0bb050bc7a49dd28d501f5bd96282458ad218a7bf" exitCode=0 Jul 14 17:33:43.585654 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:43.585581 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09znmf" event={"ID":"1bf1435a-7689-4366-879b-562c985f0d65","Type":"ContainerDied","Data":"0a00e025a31be4e137266fb0bb050bc7a49dd28d501f5bd96282458ad218a7bf"} Jul 14 17:33:44.592521 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:44.592484 2554 generic.go:356] "Generic (PLEG): container finished" podID="0045d523-1b12-4f86-bc45-e490ec45a739" containerID="6b3f30a61f75b145367d845327ba84c3ad52848ccdf078e2b6e0c0c4f6aa1e97" exitCode=0 Jul 14 17:33:44.592862 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:44.592565 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wcnzk" event={"ID":"0045d523-1b12-4f86-bc45-e490ec45a739","Type":"ContainerDied","Data":"6b3f30a61f75b145367d845327ba84c3ad52848ccdf078e2b6e0c0c4f6aa1e97"} Jul 14 17:33:44.594504 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:44.594485 2554 generic.go:356] "Generic (PLEG): container finished" podID="4e9fc54e-180e-44ec-aaa1-76638d1f30ab" containerID="7594b54ec4f115e0d05c9534127410b6492fdc1f2f773567edb5ab6f8b1d06ad" exitCode=0 Jul 14 17:33:44.594582 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:44.594547 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2rwx" event={"ID":"4e9fc54e-180e-44ec-aaa1-76638d1f30ab","Type":"ContainerDied","Data":"7594b54ec4f115e0d05c9534127410b6492fdc1f2f773567edb5ab6f8b1d06ad"} Jul 14 17:33:44.596829 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:44.596546 2554 generic.go:356] "Generic (PLEG): container finished" podID="1bf1435a-7689-4366-879b-562c985f0d65" containerID="879475e3945b50d11b4c9988aaed164f941cf7d66f5d4f708b7825a365a1bdbd" exitCode=0 Jul 14 17:33:44.596829 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:44.596577 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09znmf" event={"ID":"1bf1435a-7689-4366-879b-562c985f0d65","Type":"ContainerDied","Data":"879475e3945b50d11b4c9988aaed164f941cf7d66f5d4f708b7825a365a1bdbd"} Jul 14 17:33:45.605943 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:45.605903 2554 generic.go:356] "Generic (PLEG): container finished" podID="1f45d5d7-31ab-42d5-a787-c06a1fe43b2f" containerID="477f159c321f934c8d5649236ab8715069796b6d0d1689860165103955b9ea1b" exitCode=0 Jul 14 17:33:45.606361 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:45.605982 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mvnqq" event={"ID":"1f45d5d7-31ab-42d5-a787-c06a1fe43b2f","Type":"ContainerDied","Data":"477f159c321f934c8d5649236ab8715069796b6d0d1689860165103955b9ea1b"} Jul 14 17:33:45.754788 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:45.754760 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09znmf" Jul 14 17:33:45.835745 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:45.835722 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wcnzk" Jul 14 17:33:45.838414 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:45.838397 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2rwx" Jul 14 17:33:45.851708 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:45.851681 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qgxxg\" (UniqueName: \"kubernetes.io/projected/1bf1435a-7689-4366-879b-562c985f0d65-kube-api-access-qgxxg\") pod \"1bf1435a-7689-4366-879b-562c985f0d65\" (UID: \"1bf1435a-7689-4366-879b-562c985f0d65\") " Jul 14 17:33:45.851833 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:45.851752 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1bf1435a-7689-4366-879b-562c985f0d65-bundle\") pod \"1bf1435a-7689-4366-879b-562c985f0d65\" (UID: \"1bf1435a-7689-4366-879b-562c985f0d65\") " Jul 14 17:33:45.851833 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:45.851781 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1bf1435a-7689-4366-879b-562c985f0d65-util\") pod \"1bf1435a-7689-4366-879b-562c985f0d65\" (UID: \"1bf1435a-7689-4366-879b-562c985f0d65\") " Jul 14 17:33:45.852992 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:45.852964 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bf1435a-7689-4366-879b-562c985f0d65-bundle" (OuterVolumeSpecName: "bundle") pod "1bf1435a-7689-4366-879b-562c985f0d65" (UID: "1bf1435a-7689-4366-879b-562c985f0d65"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:33:45.854621 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:45.854583 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bf1435a-7689-4366-879b-562c985f0d65-kube-api-access-qgxxg" (OuterVolumeSpecName: "kube-api-access-qgxxg") pod "1bf1435a-7689-4366-879b-562c985f0d65" (UID: "1bf1435a-7689-4366-879b-562c985f0d65"). InnerVolumeSpecName "kube-api-access-qgxxg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:33:45.858259 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:45.858190 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bf1435a-7689-4366-879b-562c985f0d65-util" (OuterVolumeSpecName: "util") pod "1bf1435a-7689-4366-879b-562c985f0d65" (UID: "1bf1435a-7689-4366-879b-562c985f0d65"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:33:45.953017 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:45.952986 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tzkj9\" (UniqueName: \"kubernetes.io/projected/0045d523-1b12-4f86-bc45-e490ec45a739-kube-api-access-tzkj9\") pod \"0045d523-1b12-4f86-bc45-e490ec45a739\" (UID: \"0045d523-1b12-4f86-bc45-e490ec45a739\") " Jul 14 17:33:45.953189 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:45.953048 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4e9fc54e-180e-44ec-aaa1-76638d1f30ab-bundle\") pod \"4e9fc54e-180e-44ec-aaa1-76638d1f30ab\" (UID: \"4e9fc54e-180e-44ec-aaa1-76638d1f30ab\") " Jul 14 17:33:45.953189 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:45.953078 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0045d523-1b12-4f86-bc45-e490ec45a739-bundle\") pod \"0045d523-1b12-4f86-bc45-e490ec45a739\" (UID: \"0045d523-1b12-4f86-bc45-e490ec45a739\") " Jul 14 17:33:45.953189 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:45.953092 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4e9fc54e-180e-44ec-aaa1-76638d1f30ab-util\") pod \"4e9fc54e-180e-44ec-aaa1-76638d1f30ab\" (UID: \"4e9fc54e-180e-44ec-aaa1-76638d1f30ab\") " Jul 14 17:33:45.953189 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:45.953128 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0045d523-1b12-4f86-bc45-e490ec45a739-util\") pod \"0045d523-1b12-4f86-bc45-e490ec45a739\" (UID: \"0045d523-1b12-4f86-bc45-e490ec45a739\") " Jul 14 17:33:45.953189 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:45.953148 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jzd8l\" (UniqueName: \"kubernetes.io/projected/4e9fc54e-180e-44ec-aaa1-76638d1f30ab-kube-api-access-jzd8l\") pod \"4e9fc54e-180e-44ec-aaa1-76638d1f30ab\" (UID: \"4e9fc54e-180e-44ec-aaa1-76638d1f30ab\") " Jul 14 17:33:45.953471 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:45.953358 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgxxg\" (UniqueName: \"kubernetes.io/projected/1bf1435a-7689-4366-879b-562c985f0d65-kube-api-access-qgxxg\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:33:45.953471 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:45.953378 2554 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1bf1435a-7689-4366-879b-562c985f0d65-bundle\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:33:45.953471 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:45.953390 2554 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1bf1435a-7689-4366-879b-562c985f0d65-util\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:33:45.953752 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:45.953722 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e9fc54e-180e-44ec-aaa1-76638d1f30ab-bundle" (OuterVolumeSpecName: "bundle") pod "4e9fc54e-180e-44ec-aaa1-76638d1f30ab" (UID: "4e9fc54e-180e-44ec-aaa1-76638d1f30ab"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:33:45.953836 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:45.953802 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0045d523-1b12-4f86-bc45-e490ec45a739-bundle" (OuterVolumeSpecName: "bundle") pod "0045d523-1b12-4f86-bc45-e490ec45a739" (UID: "0045d523-1b12-4f86-bc45-e490ec45a739"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:33:45.955171 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:45.955149 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e9fc54e-180e-44ec-aaa1-76638d1f30ab-kube-api-access-jzd8l" (OuterVolumeSpecName: "kube-api-access-jzd8l") pod "4e9fc54e-180e-44ec-aaa1-76638d1f30ab" (UID: "4e9fc54e-180e-44ec-aaa1-76638d1f30ab"). InnerVolumeSpecName "kube-api-access-jzd8l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:33:45.955298 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:45.955179 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0045d523-1b12-4f86-bc45-e490ec45a739-kube-api-access-tzkj9" (OuterVolumeSpecName: "kube-api-access-tzkj9") pod "0045d523-1b12-4f86-bc45-e490ec45a739" (UID: "0045d523-1b12-4f86-bc45-e490ec45a739"). InnerVolumeSpecName "kube-api-access-tzkj9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:33:45.958748 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:45.958717 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e9fc54e-180e-44ec-aaa1-76638d1f30ab-util" (OuterVolumeSpecName: "util") pod "4e9fc54e-180e-44ec-aaa1-76638d1f30ab" (UID: "4e9fc54e-180e-44ec-aaa1-76638d1f30ab"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:33:46.054013 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:46.053979 2554 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4e9fc54e-180e-44ec-aaa1-76638d1f30ab-bundle\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:33:46.054013 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:46.054006 2554 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0045d523-1b12-4f86-bc45-e490ec45a739-bundle\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:33:46.054013 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:46.054016 2554 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4e9fc54e-180e-44ec-aaa1-76638d1f30ab-util\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:33:46.054241 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:46.054023 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzd8l\" (UniqueName: \"kubernetes.io/projected/4e9fc54e-180e-44ec-aaa1-76638d1f30ab-kube-api-access-jzd8l\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:33:46.054241 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:46.054034 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzkj9\" (UniqueName: \"kubernetes.io/projected/0045d523-1b12-4f86-bc45-e490ec45a739-kube-api-access-tzkj9\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:33:46.152263 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:46.152169 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0045d523-1b12-4f86-bc45-e490ec45a739-util" (OuterVolumeSpecName: "util") pod "0045d523-1b12-4f86-bc45-e490ec45a739" (UID: "0045d523-1b12-4f86-bc45-e490ec45a739"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:33:46.155346 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:46.155323 2554 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0045d523-1b12-4f86-bc45-e490ec45a739-util\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:33:46.613092 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:46.613055 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2rwx" event={"ID":"4e9fc54e-180e-44ec-aaa1-76638d1f30ab","Type":"ContainerDied","Data":"53859df757313cf4e3056d9eda9f3c3dd0997e370df5df96c735a8ef8b214941"} Jul 14 17:33:46.613092 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:46.613087 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2rwx" Jul 14 17:33:46.613559 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:46.613125 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="53859df757313cf4e3056d9eda9f3c3dd0997e370df5df96c735a8ef8b214941" Jul 14 17:33:46.615321 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:46.615280 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09znmf" event={"ID":"1bf1435a-7689-4366-879b-562c985f0d65","Type":"ContainerDied","Data":"77e759050f582198df1e20d57860f45461159433388e5f99b251a7351e42d74d"} Jul 14 17:33:46.615321 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:46.615319 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="77e759050f582198df1e20d57860f45461159433388e5f99b251a7351e42d74d" Jul 14 17:33:46.615599 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:46.615559 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09znmf" Jul 14 17:33:46.617632 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:46.617611 2554 generic.go:356] "Generic (PLEG): container finished" podID="1f45d5d7-31ab-42d5-a787-c06a1fe43b2f" containerID="8f1eca86197a017b3a480f9c8b6cd330a6d39588d5b9145a4075d240d1ab853d" exitCode=0 Jul 14 17:33:46.617740 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:46.617689 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mvnqq" event={"ID":"1f45d5d7-31ab-42d5-a787-c06a1fe43b2f","Type":"ContainerDied","Data":"8f1eca86197a017b3a480f9c8b6cd330a6d39588d5b9145a4075d240d1ab853d"} Jul 14 17:33:46.619508 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:46.619493 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wcnzk" Jul 14 17:33:46.619589 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:46.619488 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wcnzk" event={"ID":"0045d523-1b12-4f86-bc45-e490ec45a739","Type":"ContainerDied","Data":"34f9dc1951a2dd520df7ff077c85a20969091753f14b88b3cfdb40371a2aaf69"} Jul 14 17:33:46.619628 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:46.619606 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="34f9dc1951a2dd520df7ff077c85a20969091753f14b88b3cfdb40371a2aaf69" Jul 14 17:33:47.749763 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:47.749739 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mvnqq" Jul 14 17:33:47.871723 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:47.871692 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sktdk\" (UniqueName: \"kubernetes.io/projected/1f45d5d7-31ab-42d5-a787-c06a1fe43b2f-kube-api-access-sktdk\") pod \"1f45d5d7-31ab-42d5-a787-c06a1fe43b2f\" (UID: \"1f45d5d7-31ab-42d5-a787-c06a1fe43b2f\") " Jul 14 17:33:47.871723 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:47.871725 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1f45d5d7-31ab-42d5-a787-c06a1fe43b2f-util\") pod \"1f45d5d7-31ab-42d5-a787-c06a1fe43b2f\" (UID: \"1f45d5d7-31ab-42d5-a787-c06a1fe43b2f\") " Jul 14 17:33:47.871933 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:47.871780 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1f45d5d7-31ab-42d5-a787-c06a1fe43b2f-bundle\") pod \"1f45d5d7-31ab-42d5-a787-c06a1fe43b2f\" (UID: \"1f45d5d7-31ab-42d5-a787-c06a1fe43b2f\") " Jul 14 17:33:47.872305 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:47.872267 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f45d5d7-31ab-42d5-a787-c06a1fe43b2f-bundle" (OuterVolumeSpecName: "bundle") pod "1f45d5d7-31ab-42d5-a787-c06a1fe43b2f" (UID: "1f45d5d7-31ab-42d5-a787-c06a1fe43b2f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:33:47.873723 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:47.873700 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f45d5d7-31ab-42d5-a787-c06a1fe43b2f-kube-api-access-sktdk" (OuterVolumeSpecName: "kube-api-access-sktdk") pod "1f45d5d7-31ab-42d5-a787-c06a1fe43b2f" (UID: "1f45d5d7-31ab-42d5-a787-c06a1fe43b2f"). InnerVolumeSpecName "kube-api-access-sktdk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:33:47.876876 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:47.876832 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f45d5d7-31ab-42d5-a787-c06a1fe43b2f-util" (OuterVolumeSpecName: "util") pod "1f45d5d7-31ab-42d5-a787-c06a1fe43b2f" (UID: "1f45d5d7-31ab-42d5-a787-c06a1fe43b2f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:33:47.972856 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:47.972810 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sktdk\" (UniqueName: \"kubernetes.io/projected/1f45d5d7-31ab-42d5-a787-c06a1fe43b2f-kube-api-access-sktdk\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:33:47.972856 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:47.972847 2554 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1f45d5d7-31ab-42d5-a787-c06a1fe43b2f-util\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:33:47.972856 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:47.972860 2554 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1f45d5d7-31ab-42d5-a787-c06a1fe43b2f-bundle\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:33:48.631883 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:48.631855 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mvnqq" Jul 14 17:33:48.632034 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:48.631882 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mvnqq" event={"ID":"1f45d5d7-31ab-42d5-a787-c06a1fe43b2f","Type":"ContainerDied","Data":"885980252fdac403d015bcb16623917234efdc7c068c19963c20ae2b55a41d11"} Jul 14 17:33:48.632034 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:48.631933 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="885980252fdac403d015bcb16623917234efdc7c068c19963c20ae2b55a41d11" Jul 14 17:33:49.339475 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:49.339440 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5b6ff48bc7-dss8d" Jul 14 17:33:49.339882 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:49.339497 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5b6ff48bc7-dss8d" Jul 14 17:33:49.344175 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:49.344149 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5b6ff48bc7-dss8d" Jul 14 17:33:49.643396 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:49.643370 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5b6ff48bc7-dss8d" Jul 14 17:33:49.695429 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:33:49.695396 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c58849875-fbddm"] Jul 14 17:34:14.715623 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:34:14.715583 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7c58849875-fbddm" podUID="c77f9f28-61ed-4eb8-b9cb-205bc21d38db" containerName="console" containerID="cri-o://f4aef61bbb53aacdbc3a4faa3917893e7a17f172505d75dd9ce21620c7de20fc" gracePeriod=15 Jul 14 17:34:14.964520 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:34:14.964495 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c58849875-fbddm" Jul 14 17:34:15.016924 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:34:15.016855 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-trusted-ca-bundle\") pod \"c77f9f28-61ed-4eb8-b9cb-205bc21d38db\" (UID: \"c77f9f28-61ed-4eb8-b9cb-205bc21d38db\") " Jul 14 17:34:15.016924 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:34:15.016890 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-console-config\") pod \"c77f9f28-61ed-4eb8-b9cb-205bc21d38db\" (UID: \"c77f9f28-61ed-4eb8-b9cb-205bc21d38db\") " Jul 14 17:34:15.017109 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:34:15.016938 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-service-ca\") pod \"c77f9f28-61ed-4eb8-b9cb-205bc21d38db\" (UID: \"c77f9f28-61ed-4eb8-b9cb-205bc21d38db\") " Jul 14 17:34:15.017164 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:34:15.017140 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r79h6\" (UniqueName: \"kubernetes.io/projected/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-kube-api-access-r79h6\") pod \"c77f9f28-61ed-4eb8-b9cb-205bc21d38db\" (UID: \"c77f9f28-61ed-4eb8-b9cb-205bc21d38db\") " Jul 14 17:34:15.017242 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:34:15.017188 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-console-serving-cert\") pod \"c77f9f28-61ed-4eb8-b9cb-205bc21d38db\" (UID: \"c77f9f28-61ed-4eb8-b9cb-205bc21d38db\") " Jul 14 17:34:15.017303 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:34:15.017243 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-console-oauth-config\") pod \"c77f9f28-61ed-4eb8-b9cb-205bc21d38db\" (UID: \"c77f9f28-61ed-4eb8-b9cb-205bc21d38db\") " Jul 14 17:34:15.017303 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:34:15.017268 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-service-ca" (OuterVolumeSpecName: "service-ca") pod "c77f9f28-61ed-4eb8-b9cb-205bc21d38db" (UID: "c77f9f28-61ed-4eb8-b9cb-205bc21d38db"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:34:15.017303 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:34:15.017287 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-oauth-serving-cert\") pod \"c77f9f28-61ed-4eb8-b9cb-205bc21d38db\" (UID: \"c77f9f28-61ed-4eb8-b9cb-205bc21d38db\") " Jul 14 17:34:15.017303 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:34:15.017290 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-console-config" (OuterVolumeSpecName: "console-config") pod "c77f9f28-61ed-4eb8-b9cb-205bc21d38db" (UID: "c77f9f28-61ed-4eb8-b9cb-205bc21d38db"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:34:15.017477 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:34:15.017303 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c77f9f28-61ed-4eb8-b9cb-205bc21d38db" (UID: "c77f9f28-61ed-4eb8-b9cb-205bc21d38db"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:34:15.017585 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:34:15.017569 2554 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-service-ca\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:34:15.017646 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:34:15.017590 2554 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-trusted-ca-bundle\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:34:15.017646 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:34:15.017604 2554 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-console-config\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:34:15.017723 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:34:15.017666 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c77f9f28-61ed-4eb8-b9cb-205bc21d38db" (UID: "c77f9f28-61ed-4eb8-b9cb-205bc21d38db"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:34:15.019207 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:34:15.019184 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c77f9f28-61ed-4eb8-b9cb-205bc21d38db" (UID: "c77f9f28-61ed-4eb8-b9cb-205bc21d38db"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:34:15.019655 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:34:15.019635 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-kube-api-access-r79h6" (OuterVolumeSpecName: "kube-api-access-r79h6") pod "c77f9f28-61ed-4eb8-b9cb-205bc21d38db" (UID: "c77f9f28-61ed-4eb8-b9cb-205bc21d38db"). InnerVolumeSpecName "kube-api-access-r79h6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:34:15.019730 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:34:15.019637 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c77f9f28-61ed-4eb8-b9cb-205bc21d38db" (UID: "c77f9f28-61ed-4eb8-b9cb-205bc21d38db"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:34:15.118663 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:34:15.118635 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r79h6\" (UniqueName: \"kubernetes.io/projected/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-kube-api-access-r79h6\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:34:15.118663 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:34:15.118658 2554 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-console-serving-cert\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:34:15.118663 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:34:15.118667 2554 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-console-oauth-config\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:34:15.118886 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:34:15.118677 2554 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c77f9f28-61ed-4eb8-b9cb-205bc21d38db-oauth-serving-cert\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:34:15.765153 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:34:15.765118 2554 generic.go:356] "Generic (PLEG): container finished" podID="c77f9f28-61ed-4eb8-b9cb-205bc21d38db" containerID="f4aef61bbb53aacdbc3a4faa3917893e7a17f172505d75dd9ce21620c7de20fc" exitCode=0 Jul 14 17:34:15.765568 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:34:15.765164 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c58849875-fbddm" Jul 14 17:34:15.765568 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:34:15.765180 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c58849875-fbddm" event={"ID":"c77f9f28-61ed-4eb8-b9cb-205bc21d38db","Type":"ContainerDied","Data":"f4aef61bbb53aacdbc3a4faa3917893e7a17f172505d75dd9ce21620c7de20fc"} Jul 14 17:34:15.765568 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:34:15.765235 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c58849875-fbddm" event={"ID":"c77f9f28-61ed-4eb8-b9cb-205bc21d38db","Type":"ContainerDied","Data":"da6eca2d7a1472f7acbd4f4c5d21e0f70d197d006d55c7d2fc68dd8088e801fa"} Jul 14 17:34:15.765568 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:34:15.765258 2554 scope.go:117] "RemoveContainer" containerID="f4aef61bbb53aacdbc3a4faa3917893e7a17f172505d75dd9ce21620c7de20fc" Jul 14 17:34:15.774022 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:34:15.774005 2554 scope.go:117] "RemoveContainer" containerID="f4aef61bbb53aacdbc3a4faa3917893e7a17f172505d75dd9ce21620c7de20fc" Jul 14 17:34:15.774262 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:34:15.774245 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4aef61bbb53aacdbc3a4faa3917893e7a17f172505d75dd9ce21620c7de20fc\": container with ID starting with f4aef61bbb53aacdbc3a4faa3917893e7a17f172505d75dd9ce21620c7de20fc not found: ID does not exist" containerID="f4aef61bbb53aacdbc3a4faa3917893e7a17f172505d75dd9ce21620c7de20fc" Jul 14 17:34:15.774334 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:34:15.774267 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4aef61bbb53aacdbc3a4faa3917893e7a17f172505d75dd9ce21620c7de20fc"} err="failed to get container status \"f4aef61bbb53aacdbc3a4faa3917893e7a17f172505d75dd9ce21620c7de20fc\": rpc error: code = NotFound desc = could not find container \"f4aef61bbb53aacdbc3a4faa3917893e7a17f172505d75dd9ce21620c7de20fc\": container with ID starting with f4aef61bbb53aacdbc3a4faa3917893e7a17f172505d75dd9ce21620c7de20fc not found: ID does not exist" Jul 14 17:34:15.787162 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:34:15.787138 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c58849875-fbddm"] Jul 14 17:34:15.793422 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:34:15.793392 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c58849875-fbddm"] Jul 14 17:34:16.387892 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:34:16.387861 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c77f9f28-61ed-4eb8-b9cb-205bc21d38db" path="/var/lib/kubelet/pods/c77f9f28-61ed-4eb8-b9cb-205bc21d38db/volumes" Jul 14 17:35:19.745965 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.745932 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:35:19.746408 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.746263 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e9fc54e-180e-44ec-aaa1-76638d1f30ab" containerName="extract" Jul 14 17:35:19.746408 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.746274 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e9fc54e-180e-44ec-aaa1-76638d1f30ab" containerName="extract" Jul 14 17:35:19.746408 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.746289 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0045d523-1b12-4f86-bc45-e490ec45a739" containerName="util" Jul 14 17:35:19.746408 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.746294 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="0045d523-1b12-4f86-bc45-e490ec45a739" containerName="util" Jul 14 17:35:19.746408 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.746303 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0045d523-1b12-4f86-bc45-e490ec45a739" containerName="extract" Jul 14 17:35:19.746408 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.746309 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="0045d523-1b12-4f86-bc45-e490ec45a739" containerName="extract" Jul 14 17:35:19.746408 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.746316 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c77f9f28-61ed-4eb8-b9cb-205bc21d38db" containerName="console" Jul 14 17:35:19.746408 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.746321 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="c77f9f28-61ed-4eb8-b9cb-205bc21d38db" containerName="console" Jul 14 17:35:19.746408 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.746328 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0045d523-1b12-4f86-bc45-e490ec45a739" containerName="pull" Jul 14 17:35:19.746408 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.746333 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="0045d523-1b12-4f86-bc45-e490ec45a739" containerName="pull" Jul 14 17:35:19.746408 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.746340 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bf1435a-7689-4366-879b-562c985f0d65" containerName="util" Jul 14 17:35:19.746408 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.746345 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bf1435a-7689-4366-879b-562c985f0d65" containerName="util" Jul 14 17:35:19.746408 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.746351 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bf1435a-7689-4366-879b-562c985f0d65" containerName="extract" Jul 14 17:35:19.746408 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.746356 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bf1435a-7689-4366-879b-562c985f0d65" containerName="extract" Jul 14 17:35:19.746408 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.746362 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f45d5d7-31ab-42d5-a787-c06a1fe43b2f" containerName="extract" Jul 14 17:35:19.746408 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.746366 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f45d5d7-31ab-42d5-a787-c06a1fe43b2f" containerName="extract" Jul 14 17:35:19.746408 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.746373 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e9fc54e-180e-44ec-aaa1-76638d1f30ab" containerName="pull" Jul 14 17:35:19.746408 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.746377 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e9fc54e-180e-44ec-aaa1-76638d1f30ab" containerName="pull" Jul 14 17:35:19.746408 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.746382 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bf1435a-7689-4366-879b-562c985f0d65" containerName="pull" Jul 14 17:35:19.746408 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.746387 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bf1435a-7689-4366-879b-562c985f0d65" containerName="pull" Jul 14 17:35:19.746408 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.746392 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e9fc54e-180e-44ec-aaa1-76638d1f30ab" containerName="util" Jul 14 17:35:19.746408 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.746397 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e9fc54e-180e-44ec-aaa1-76638d1f30ab" containerName="util" Jul 14 17:35:19.746408 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.746404 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f45d5d7-31ab-42d5-a787-c06a1fe43b2f" containerName="util" Jul 14 17:35:19.746408 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.746408 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f45d5d7-31ab-42d5-a787-c06a1fe43b2f" containerName="util" Jul 14 17:35:19.746408 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.746413 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f45d5d7-31ab-42d5-a787-c06a1fe43b2f" containerName="pull" Jul 14 17:35:19.746408 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.746418 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f45d5d7-31ab-42d5-a787-c06a1fe43b2f" containerName="pull" Jul 14 17:35:19.747141 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.746480 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f45d5d7-31ab-42d5-a787-c06a1fe43b2f" containerName="extract" Jul 14 17:35:19.747141 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.746489 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="c77f9f28-61ed-4eb8-b9cb-205bc21d38db" containerName="console" Jul 14 17:35:19.747141 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.746494 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="1bf1435a-7689-4366-879b-562c985f0d65" containerName="extract" Jul 14 17:35:19.747141 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.746500 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e9fc54e-180e-44ec-aaa1-76638d1f30ab" containerName="extract" Jul 14 17:35:19.747141 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.746506 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="0045d523-1b12-4f86-bc45-e490ec45a739" containerName="extract" Jul 14 17:35:19.749334 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.749318 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-swz27" Jul 14 17:35:19.752187 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.752167 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-wbl4l\"" Jul 14 17:35:19.752382 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.752361 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 14 17:35:19.753265 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.753245 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 14 17:35:19.753353 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.753259 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 14 17:35:19.768069 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.768047 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:35:19.806286 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.806264 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:35:19.860016 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.859986 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9rc4\" (UniqueName: \"kubernetes.io/projected/ffbc8fa7-8283-4f2b-9e46-724ef9ac1c41-kube-api-access-s9rc4\") pod \"limitador-limitador-54d6d988d5-swz27\" (UID: \"ffbc8fa7-8283-4f2b-9e46-724ef9ac1c41\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-swz27" Jul 14 17:35:19.860127 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.860018 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ffbc8fa7-8283-4f2b-9e46-724ef9ac1c41-config-file\") pod \"limitador-limitador-54d6d988d5-swz27\" (UID: \"ffbc8fa7-8283-4f2b-9e46-724ef9ac1c41\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-swz27" Jul 14 17:35:19.960509 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.960480 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9rc4\" (UniqueName: \"kubernetes.io/projected/ffbc8fa7-8283-4f2b-9e46-724ef9ac1c41-kube-api-access-s9rc4\") pod \"limitador-limitador-54d6d988d5-swz27\" (UID: \"ffbc8fa7-8283-4f2b-9e46-724ef9ac1c41\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-swz27" Jul 14 17:35:19.960627 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.960510 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ffbc8fa7-8283-4f2b-9e46-724ef9ac1c41-config-file\") pod \"limitador-limitador-54d6d988d5-swz27\" (UID: \"ffbc8fa7-8283-4f2b-9e46-724ef9ac1c41\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-swz27" Jul 14 17:35:19.961070 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.961054 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ffbc8fa7-8283-4f2b-9e46-724ef9ac1c41-config-file\") pod \"limitador-limitador-54d6d988d5-swz27\" (UID: \"ffbc8fa7-8283-4f2b-9e46-724ef9ac1c41\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-swz27" Jul 14 17:35:19.969793 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:19.969764 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9rc4\" (UniqueName: \"kubernetes.io/projected/ffbc8fa7-8283-4f2b-9e46-724ef9ac1c41-kube-api-access-s9rc4\") pod \"limitador-limitador-54d6d988d5-swz27\" (UID: \"ffbc8fa7-8283-4f2b-9e46-724ef9ac1c41\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-swz27" Jul 14 17:35:20.058850 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:20.058766 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-swz27" Jul 14 17:35:20.193712 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:20.193685 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:35:20.195430 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:35:20.195404 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffbc8fa7_8283_4f2b_9e46_724ef9ac1c41.slice/crio-16f32719377ed6e8589ce06e0d76cafd2540f1671923bebf5bb9d527442caf27 WatchSource:0}: Error finding container 16f32719377ed6e8589ce06e0d76cafd2540f1671923bebf5bb9d527442caf27: Status 404 returned error can't find the container with id 16f32719377ed6e8589ce06e0d76cafd2540f1671923bebf5bb9d527442caf27 Jul 14 17:35:21.094229 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:21.094185 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-swz27" event={"ID":"ffbc8fa7-8283-4f2b-9e46-724ef9ac1c41","Type":"ContainerStarted","Data":"16f32719377ed6e8589ce06e0d76cafd2540f1671923bebf5bb9d527442caf27"} Jul 14 17:35:23.107931 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:23.107890 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-swz27" event={"ID":"ffbc8fa7-8283-4f2b-9e46-724ef9ac1c41","Type":"ContainerStarted","Data":"1ff37c44e12c3855e8006f9884fb893c8de1c51cac41c4d55141686038c94b11"} Jul 14 17:35:23.108394 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:23.108009 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-swz27" Jul 14 17:35:23.127341 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:23.127295 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-swz27" podStartSLOduration=1.422054098 podStartE2EDuration="4.127281688s" totalImagesPullingTime="2.70522759s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:35:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:35:23.124628626 +0000 UTC m=+567.265877373" watchObservedRunningTime="2026-07-14 17:35:23.127281688 +0000 UTC m=+567.268530392" Jul 14 17:35:34.113317 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:34.113289 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-swz27" Jul 14 17:35:56.263382 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:56.263351 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pk9h2_f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726/ovn-acl-logging/0.log" Jul 14 17:35:56.264598 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:35:56.264576 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pk9h2_f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726/ovn-acl-logging/0.log" Jul 14 17:36:30.785876 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:36:30.785782 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:36:59.951155 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:36:59.951119 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:37:15.245525 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:37:15.245488 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:37:19.682420 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:37:19.682385 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-pmzkj"] Jul 14 17:37:19.684604 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:37:19.684580 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pmzkj" Jul 14 17:37:19.688397 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:37:19.688377 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-w8snq\"" Jul 14 17:37:19.688524 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:37:19.688379 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 14 17:37:19.688524 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:37:19.688381 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 14 17:37:19.688524 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:37:19.688439 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 14 17:37:19.696900 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:37:19.696877 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-pmzkj"] Jul 14 17:37:19.732108 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:37:19.732078 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c107c170-61cc-46a8-bc76-bef865a6f3b7-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pmzkj\" (UID: \"c107c170-61cc-46a8-bc76-bef865a6f3b7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pmzkj" Jul 14 17:37:19.732246 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:37:19.732134 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwwcp\" (UniqueName: \"kubernetes.io/projected/c107c170-61cc-46a8-bc76-bef865a6f3b7-kube-api-access-nwwcp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pmzkj\" (UID: \"c107c170-61cc-46a8-bc76-bef865a6f3b7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pmzkj" Jul 14 17:37:19.732246 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:37:19.732176 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c107c170-61cc-46a8-bc76-bef865a6f3b7-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pmzkj\" (UID: \"c107c170-61cc-46a8-bc76-bef865a6f3b7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pmzkj" Jul 14 17:37:19.732246 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:37:19.732205 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c107c170-61cc-46a8-bc76-bef865a6f3b7-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pmzkj\" (UID: \"c107c170-61cc-46a8-bc76-bef865a6f3b7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pmzkj" Jul 14 17:37:19.732371 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:37:19.732255 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c107c170-61cc-46a8-bc76-bef865a6f3b7-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pmzkj\" (UID: \"c107c170-61cc-46a8-bc76-bef865a6f3b7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pmzkj" Jul 14 17:37:19.732371 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:37:19.732299 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c107c170-61cc-46a8-bc76-bef865a6f3b7-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pmzkj\" (UID: \"c107c170-61cc-46a8-bc76-bef865a6f3b7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pmzkj" Jul 14 17:37:19.832818 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:37:19.832791 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c107c170-61cc-46a8-bc76-bef865a6f3b7-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pmzkj\" (UID: \"c107c170-61cc-46a8-bc76-bef865a6f3b7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pmzkj" Jul 14 17:37:19.832818 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:37:19.832823 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c107c170-61cc-46a8-bc76-bef865a6f3b7-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pmzkj\" (UID: \"c107c170-61cc-46a8-bc76-bef865a6f3b7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pmzkj" Jul 14 17:37:19.833036 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:37:19.832862 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c107c170-61cc-46a8-bc76-bef865a6f3b7-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pmzkj\" (UID: \"c107c170-61cc-46a8-bc76-bef865a6f3b7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pmzkj" Jul 14 17:37:19.833036 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:37:19.832901 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c107c170-61cc-46a8-bc76-bef865a6f3b7-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pmzkj\" (UID: \"c107c170-61cc-46a8-bc76-bef865a6f3b7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pmzkj" Jul 14 17:37:19.833036 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:37:19.832934 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwwcp\" (UniqueName: \"kubernetes.io/projected/c107c170-61cc-46a8-bc76-bef865a6f3b7-kube-api-access-nwwcp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pmzkj\" (UID: \"c107c170-61cc-46a8-bc76-bef865a6f3b7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pmzkj" Jul 14 17:37:19.833036 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:37:19.832963 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c107c170-61cc-46a8-bc76-bef865a6f3b7-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pmzkj\" (UID: \"c107c170-61cc-46a8-bc76-bef865a6f3b7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pmzkj" Jul 14 17:37:19.833329 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:37:19.833309 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c107c170-61cc-46a8-bc76-bef865a6f3b7-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pmzkj\" (UID: \"c107c170-61cc-46a8-bc76-bef865a6f3b7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pmzkj" Jul 14 17:37:19.833425 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:37:19.833337 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c107c170-61cc-46a8-bc76-bef865a6f3b7-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pmzkj\" (UID: \"c107c170-61cc-46a8-bc76-bef865a6f3b7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pmzkj" Jul 14 17:37:19.833425 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:37:19.833380 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c107c170-61cc-46a8-bc76-bef865a6f3b7-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pmzkj\" (UID: \"c107c170-61cc-46a8-bc76-bef865a6f3b7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pmzkj" Jul 14 17:37:19.835170 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:37:19.835149 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c107c170-61cc-46a8-bc76-bef865a6f3b7-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pmzkj\" (UID: \"c107c170-61cc-46a8-bc76-bef865a6f3b7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pmzkj" Jul 14 17:37:19.835462 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:37:19.835443 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c107c170-61cc-46a8-bc76-bef865a6f3b7-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pmzkj\" (UID: \"c107c170-61cc-46a8-bc76-bef865a6f3b7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pmzkj" Jul 14 17:37:19.841811 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:37:19.841788 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwwcp\" (UniqueName: \"kubernetes.io/projected/c107c170-61cc-46a8-bc76-bef865a6f3b7-kube-api-access-nwwcp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pmzkj\" (UID: \"c107c170-61cc-46a8-bc76-bef865a6f3b7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pmzkj" Jul 14 17:37:19.995406 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:37:19.995302 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pmzkj" Jul 14 17:37:20.122830 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:37:20.122803 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-pmzkj"] Jul 14 17:37:20.124437 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:37:20.124412 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc107c170_61cc_46a8_bc76_bef865a6f3b7.slice/crio-02b6eea8c367a064bde6fc82857b725f2bd564e96fc8882983db5a93648ff947 WatchSource:0}: Error finding container 02b6eea8c367a064bde6fc82857b725f2bd564e96fc8882983db5a93648ff947: Status 404 returned error can't find the container with id 02b6eea8c367a064bde6fc82857b725f2bd564e96fc8882983db5a93648ff947 Jul 14 17:37:20.125800 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:37:20.125784 2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 17:37:20.702696 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:37:20.702657 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pmzkj" event={"ID":"c107c170-61cc-46a8-bc76-bef865a6f3b7","Type":"ContainerStarted","Data":"02b6eea8c367a064bde6fc82857b725f2bd564e96fc8882983db5a93648ff947"} Jul 14 17:37:20.849193 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:37:20.849159 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:37:21.709725 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:37:21.709688 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pmzkj" event={"ID":"c107c170-61cc-46a8-bc76-bef865a6f3b7","Type":"ContainerStarted","Data":"7e37ba5708583987f593e1e52fc8a2db7bef8e864cedd5299f1e9489a4074b9e"} Jul 14 17:37:21.735724 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:37:21.735663 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pmzkj" podStartSLOduration=1.719184511 podStartE2EDuration="2.735642403s" totalImagesPullingTime="1.016457892s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:37:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:37:21.733480097 +0000 UTC m=+685.874728807" watchObservedRunningTime="2026-07-14 17:37:21.735642403 +0000 UTC m=+685.876891108" Jul 14 17:37:29.996123 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:37:29.996073 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pmzkj" Jul 14 17:37:29.996556 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:37:29.996244 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pmzkj" Jul 14 17:37:30.008480 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:37:30.008458 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pmzkj" Jul 14 17:37:30.764376 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:37:30.764344 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pmzkj" Jul 14 17:37:45.342474 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:37:45.342439 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:38:07.343730 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:38:07.343648 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:38:29.244223 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:38:29.244176 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:40:04.550900 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:40:04.550857 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:40:14.040320 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:40:14.040287 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:40:24.538663 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:40:24.538626 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:40:34.912517 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:40:34.912475 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:40:44.940643 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:40:44.940609 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:40:56.297460 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:40:56.297400 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pk9h2_f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726/ovn-acl-logging/0.log" Jul 14 17:40:56.297991 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:40:56.297476 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pk9h2_f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726/ovn-acl-logging/0.log" Jul 14 17:41:10.941398 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:41:10.941357 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:41:21.738155 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:41:21.738115 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:41:57.041357 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:41:57.041318 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:42:05.748355 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:42:05.748311 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:42:15.744960 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:42:15.744925 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:42:21.351275 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:42:21.351180 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:42:32.939227 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:42:32.939182 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:42:44.441262 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:42:44.441206 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:42:55.243898 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:42:55.243861 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:43:05.549144 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:43:05.549111 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:43:31.038083 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:43:31.038042 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:43:40.238472 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:43:40.236530 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:44:03.847186 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:44:03.847108 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:44:14.939571 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:44:14.939536 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:44:28.950388 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:44:28.950354 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:44:39.655668 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:44:39.655629 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:44:49.355969 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:44:49.355938 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:44:58.939584 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:44:58.939545 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:45:09.956133 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:45:09.956096 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:45:56.331439 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:45:56.331390 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pk9h2_f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726/ovn-acl-logging/0.log" Jul 14 17:45:56.332134 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:45:56.332116 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pk9h2_f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726/ovn-acl-logging/0.log" Jul 14 17:46:57.648842 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:46:57.648804 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:47:09.240844 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:47:09.240810 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:47:17.738810 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:47:17.738778 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:47:28.343757 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:47:28.343717 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:47:38.644121 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:47:38.644083 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:47:49.444313 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:47:49.444277 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:47:58.347715 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:47:58.347670 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:48:08.741905 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:48:08.741858 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:48:19.148447 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:48:19.148367 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:48:27.049285 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:48:27.049248 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:48:36.844870 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:48:36.844824 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:48:45.145665 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:48:45.145634 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:49:03.848870 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:49:03.848832 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:49:13.152500 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:49:13.152462 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:49:22.647998 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:49:22.647962 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:49:34.961855 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:49:34.961819 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:49:52.241117 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:49:52.241034 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:50:01.645562 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:50:01.645526 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:50:10.956363 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:50:10.956325 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:50:21.360546 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:50:21.360501 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:50:30.747424 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:50:30.747389 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:50:40.270097 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:50:40.270055 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:50:53.267405 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:50:53.267369 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:50:56.363531 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:50:56.363505 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pk9h2_f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726/ovn-acl-logging/0.log" Jul 14 17:50:56.366179 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:50:56.366151 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pk9h2_f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726/ovn-acl-logging/0.log" Jul 14 17:51:02.767894 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:51:02.767857 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:51:12.503747 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:51:12.503712 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:51:24.844850 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:51:24.844769 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:51:33.850255 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:51:33.850199 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:51:40.760600 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:51:40.760559 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:51:56.646095 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:51:56.646068 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:52:06.956079 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:52:06.956044 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:52:18.452967 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:52:18.452931 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:52:31.255471 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:52:31.255432 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:52:45.443749 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:52:45.443707 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:52:52.364452 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:52:52.364419 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-06d94c77-5b4978494b-tmjw8"] Jul 14 17:52:52.368048 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:52:52.368030 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-06d94c77-5b4978494b-tmjw8" Jul 14 17:52:52.370838 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:52:52.370810 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-ggvwm\"" Jul 14 17:52:52.371094 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:52:52.371079 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-06d94c77-serving-cert\"" Jul 14 17:52:52.372018 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:52:52.372002 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 14 17:52:52.372157 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:52:52.372015 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 17:52:52.378678 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:52:52.378656 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-06d94c77-5b4978494b-tmjw8"] Jul 14 17:52:52.494421 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:52:52.494384 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxlc2\" (UniqueName: \"kubernetes.io/projected/442fcfc8-9d12-4193-8c8a-faeb59304021-kube-api-access-lxlc2\") pod \"maas-api-e2e-ait-06d94c77-5b4978494b-tmjw8\" (UID: \"442fcfc8-9d12-4193-8c8a-faeb59304021\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-06d94c77-5b4978494b-tmjw8" Jul 14 17:52:52.494421 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:52:52.494427 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/442fcfc8-9d12-4193-8c8a-faeb59304021-openshift-service-ca\") pod \"maas-api-e2e-ait-06d94c77-5b4978494b-tmjw8\" (UID: \"442fcfc8-9d12-4193-8c8a-faeb59304021\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-06d94c77-5b4978494b-tmjw8" Jul 14 17:52:52.494627 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:52:52.494449 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/442fcfc8-9d12-4193-8c8a-faeb59304021-maas-api-tls\") pod \"maas-api-e2e-ait-06d94c77-5b4978494b-tmjw8\" (UID: \"442fcfc8-9d12-4193-8c8a-faeb59304021\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-06d94c77-5b4978494b-tmjw8" Jul 14 17:52:52.595673 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:52:52.595638 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxlc2\" (UniqueName: \"kubernetes.io/projected/442fcfc8-9d12-4193-8c8a-faeb59304021-kube-api-access-lxlc2\") pod \"maas-api-e2e-ait-06d94c77-5b4978494b-tmjw8\" (UID: \"442fcfc8-9d12-4193-8c8a-faeb59304021\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-06d94c77-5b4978494b-tmjw8" Jul 14 17:52:52.595673 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:52:52.595672 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/442fcfc8-9d12-4193-8c8a-faeb59304021-openshift-service-ca\") pod \"maas-api-e2e-ait-06d94c77-5b4978494b-tmjw8\" (UID: \"442fcfc8-9d12-4193-8c8a-faeb59304021\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-06d94c77-5b4978494b-tmjw8" Jul 14 17:52:52.595864 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:52:52.595697 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/442fcfc8-9d12-4193-8c8a-faeb59304021-maas-api-tls\") pod \"maas-api-e2e-ait-06d94c77-5b4978494b-tmjw8\" (UID: \"442fcfc8-9d12-4193-8c8a-faeb59304021\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-06d94c77-5b4978494b-tmjw8" Jul 14 17:52:52.596376 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:52:52.596355 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/442fcfc8-9d12-4193-8c8a-faeb59304021-openshift-service-ca\") pod \"maas-api-e2e-ait-06d94c77-5b4978494b-tmjw8\" (UID: \"442fcfc8-9d12-4193-8c8a-faeb59304021\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-06d94c77-5b4978494b-tmjw8" Jul 14 17:52:52.598258 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:52:52.598237 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/442fcfc8-9d12-4193-8c8a-faeb59304021-maas-api-tls\") pod \"maas-api-e2e-ait-06d94c77-5b4978494b-tmjw8\" (UID: \"442fcfc8-9d12-4193-8c8a-faeb59304021\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-06d94c77-5b4978494b-tmjw8" Jul 14 17:52:52.604621 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:52:52.604597 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxlc2\" (UniqueName: \"kubernetes.io/projected/442fcfc8-9d12-4193-8c8a-faeb59304021-kube-api-access-lxlc2\") pod \"maas-api-e2e-ait-06d94c77-5b4978494b-tmjw8\" (UID: \"442fcfc8-9d12-4193-8c8a-faeb59304021\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-06d94c77-5b4978494b-tmjw8" Jul 14 17:52:52.681319 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:52:52.681237 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-06d94c77-5b4978494b-tmjw8" Jul 14 17:52:52.833405 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:52:52.833379 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-06d94c77-5b4978494b-tmjw8"] Jul 14 17:52:52.834384 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:52:52.834357 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod442fcfc8_9d12_4193_8c8a_faeb59304021.slice/crio-8ba0b373d55c2b1486b40af90f3f08ec1e7460680679044b527b794df6841202 WatchSource:0}: Error finding container 8ba0b373d55c2b1486b40af90f3f08ec1e7460680679044b527b794df6841202: Status 404 returned error can't find the container with id 8ba0b373d55c2b1486b40af90f3f08ec1e7460680679044b527b794df6841202 Jul 14 17:52:52.836121 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:52:52.836104 2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 17:52:53.371736 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:52:53.371702 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-06d94c77-5b4978494b-tmjw8" event={"ID":"442fcfc8-9d12-4193-8c8a-faeb59304021","Type":"ContainerStarted","Data":"8ba0b373d55c2b1486b40af90f3f08ec1e7460680679044b527b794df6841202"} Jul 14 17:52:55.247009 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:52:55.246975 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:52:57.273681 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:52:57.273640 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-06d94c77-jvsg7"] Jul 14 17:52:57.277065 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:52:57.277042 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-06d94c77-jvsg7" Jul 14 17:52:57.279567 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:52:57.279547 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9rnpk\"" Jul 14 17:52:57.285097 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:52:57.285076 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-06d94c77-jvsg7"] Jul 14 17:52:57.340440 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:52:57.340409 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a59f414-9cfb-47e2-923f-27aead50e7f3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-06d94c77-jvsg7\" (UID: \"2a59f414-9cfb-47e2-923f-27aead50e7f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-06d94c77-jvsg7" Jul 14 17:52:57.397191 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:52:57.397154 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-06d94c77-5b4978494b-tmjw8" event={"ID":"442fcfc8-9d12-4193-8c8a-faeb59304021","Type":"ContainerStarted","Data":"2b57a049352e7bb66c2c67e10756a3ae2279ba38e572771eadd0d613ccc0d206"} Jul 14 17:52:57.397350 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:52:57.397230 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-06d94c77-5b4978494b-tmjw8" Jul 14 17:52:57.414792 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:52:57.414747 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-06d94c77-5b4978494b-tmjw8" podStartSLOduration=1.697395049 podStartE2EDuration="5.414731286s" totalImagesPullingTime="3.717336237s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:52:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:52:57.413348515 +0000 UTC m=+1621.554597231" watchObservedRunningTime="2026-07-14 17:52:57.414731286 +0000 UTC m=+1621.555979989" Jul 14 17:52:57.441604 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:52:57.441578 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a59f414-9cfb-47e2-923f-27aead50e7f3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-06d94c77-jvsg7\" (UID: \"2a59f414-9cfb-47e2-923f-27aead50e7f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-06d94c77-jvsg7" Jul 14 17:52:57.442187 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:52:57.442171 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a59f414-9cfb-47e2-923f-27aead50e7f3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-06d94c77-jvsg7\" (UID: \"2a59f414-9cfb-47e2-923f-27aead50e7f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-06d94c77-jvsg7" Jul 14 17:52:57.589074 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:52:57.589001 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-06d94c77-jvsg7" Jul 14 17:52:57.723348 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:52:57.723321 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-06d94c77-jvsg7"] Jul 14 17:52:57.724912 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:52:57.724888 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a59f414_9cfb_47e2_923f_27aead50e7f3.slice/crio-4023ea3e76cf2dd5faba5c1d885cd114120085ff2ab1185d24d99b5c1f284b71 WatchSource:0}: Error finding container 4023ea3e76cf2dd5faba5c1d885cd114120085ff2ab1185d24d99b5c1f284b71: Status 404 returned error can't find the container with id 4023ea3e76cf2dd5faba5c1d885cd114120085ff2ab1185d24d99b5c1f284b71 Jul 14 17:52:58.403505 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:52:58.403473 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-06d94c77-jvsg7" event={"ID":"2a59f414-9cfb-47e2-923f-27aead50e7f3","Type":"ContainerStarted","Data":"4023ea3e76cf2dd5faba5c1d885cd114120085ff2ab1185d24d99b5c1f284b71"} Jul 14 17:53:00.417079 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:00.417043 2554 generic.go:356] "Generic (PLEG): container finished" podID="2a59f414-9cfb-47e2-923f-27aead50e7f3" containerID="b7d908592845fe11a84c59debe54e62f1473e16b164213f7d863427f4f033c94" exitCode=7 Jul 14 17:53:00.417434 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:00.417122 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-06d94c77-jvsg7" event={"ID":"2a59f414-9cfb-47e2-923f-27aead50e7f3","Type":"ContainerDied","Data":"b7d908592845fe11a84c59debe54e62f1473e16b164213f7d863427f4f033c94"} Jul 14 17:53:00.417434 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:00.417325 2554 scope.go:117] "RemoveContainer" containerID="b7d908592845fe11a84c59debe54e62f1473e16b164213f7d863427f4f033c94" Jul 14 17:53:01.424472 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:01.424438 2554 generic.go:356] "Generic (PLEG): container finished" podID="2a59f414-9cfb-47e2-923f-27aead50e7f3" containerID="f55677d1bfcacb41b515151a406cc99f8a1ff0e36f5fd76b5f89ebb001ac189f" exitCode=7 Jul 14 17:53:01.424867 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:01.424502 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-06d94c77-jvsg7" event={"ID":"2a59f414-9cfb-47e2-923f-27aead50e7f3","Type":"ContainerDied","Data":"f55677d1bfcacb41b515151a406cc99f8a1ff0e36f5fd76b5f89ebb001ac189f"} Jul 14 17:53:01.424867 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:01.424572 2554 scope.go:117] "RemoveContainer" containerID="b7d908592845fe11a84c59debe54e62f1473e16b164213f7d863427f4f033c94" Jul 14 17:53:01.424867 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:01.424757 2554 scope.go:117] "RemoveContainer" containerID="f55677d1bfcacb41b515151a406cc99f8a1ff0e36f5fd76b5f89ebb001ac189f" Jul 14 17:53:01.424982 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:53:01.424954 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-06d94c77-jvsg7_odh-ai-gateway-infra(2a59f414-9cfb-47e2-923f-27aead50e7f3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-06d94c77-jvsg7" podUID="2a59f414-9cfb-47e2-923f-27aead50e7f3" Jul 14 17:53:02.431421 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:02.431390 2554 scope.go:117] "RemoveContainer" containerID="f55677d1bfcacb41b515151a406cc99f8a1ff0e36f5fd76b5f89ebb001ac189f" Jul 14 17:53:02.431983 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:53:02.431645 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-06d94c77-jvsg7_odh-ai-gateway-infra(2a59f414-9cfb-47e2-923f-27aead50e7f3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-06d94c77-jvsg7" podUID="2a59f414-9cfb-47e2-923f-27aead50e7f3" Jul 14 17:53:03.409274 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:03.409245 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-06d94c77-5b4978494b-tmjw8" Jul 14 17:53:15.382841 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:15.382811 2554 scope.go:117] "RemoveContainer" containerID="f55677d1bfcacb41b515151a406cc99f8a1ff0e36f5fd76b5f89ebb001ac189f" Jul 14 17:53:16.503990 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:16.503957 2554 generic.go:356] "Generic (PLEG): container finished" podID="2a59f414-9cfb-47e2-923f-27aead50e7f3" containerID="31a97fd10d2eb18b34e285f4fe881ff4b9d7a599cb13fb753642468316a34e48" exitCode=0 Jul 14 17:53:16.504383 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:16.504031 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-06d94c77-jvsg7" event={"ID":"2a59f414-9cfb-47e2-923f-27aead50e7f3","Type":"ContainerDied","Data":"31a97fd10d2eb18b34e285f4fe881ff4b9d7a599cb13fb753642468316a34e48"} Jul 14 17:53:16.504383 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:16.504089 2554 scope.go:117] "RemoveContainer" containerID="f55677d1bfcacb41b515151a406cc99f8a1ff0e36f5fd76b5f89ebb001ac189f" Jul 14 17:53:17.535756 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:17.535727 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-06d94c77-jvsg7"] Jul 14 17:53:17.642958 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:17.642935 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-06d94c77-jvsg7" Jul 14 17:53:17.812446 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:17.812359 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a59f414-9cfb-47e2-923f-27aead50e7f3-maas-api-service-ca\") pod \"2a59f414-9cfb-47e2-923f-27aead50e7f3\" (UID: \"2a59f414-9cfb-47e2-923f-27aead50e7f3\") " Jul 14 17:53:17.812691 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:17.812669 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a59f414-9cfb-47e2-923f-27aead50e7f3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2a59f414-9cfb-47e2-923f-27aead50e7f3" (UID: "2a59f414-9cfb-47e2-923f-27aead50e7f3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:53:17.913302 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:17.913273 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a59f414-9cfb-47e2-923f-27aead50e7f3-maas-api-service-ca\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:53:18.517438 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:18.517403 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-06d94c77-jvsg7" event={"ID":"2a59f414-9cfb-47e2-923f-27aead50e7f3","Type":"ContainerDied","Data":"4023ea3e76cf2dd5faba5c1d885cd114120085ff2ab1185d24d99b5c1f284b71"} Jul 14 17:53:18.517438 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:18.517437 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-06d94c77-jvsg7" Jul 14 17:53:18.517667 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:18.517452 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4023ea3e76cf2dd5faba5c1d885cd114120085ff2ab1185d24d99b5c1f284b71" Jul 14 17:53:18.546960 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:18.546934 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-06d94c77-jvsg7"] Jul 14 17:53:18.552652 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:18.552630 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-06d94c77-jvsg7"] Jul 14 17:53:20.387127 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:20.387097 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a59f414-9cfb-47e2-923f-27aead50e7f3" path="/var/lib/kubelet/pods/2a59f414-9cfb-47e2-923f-27aead50e7f3/volumes" Jul 14 17:53:57.327739 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:57.327693 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-06d94c77-v4pk5"] Jul 14 17:53:57.328325 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:57.328079 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a59f414-9cfb-47e2-923f-27aead50e7f3" containerName="revoke-keys" Jul 14 17:53:57.328325 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:57.328091 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a59f414-9cfb-47e2-923f-27aead50e7f3" containerName="revoke-keys" Jul 14 17:53:57.328325 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:57.328110 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a59f414-9cfb-47e2-923f-27aead50e7f3" containerName="revoke-keys" Jul 14 17:53:57.328325 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:57.328115 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a59f414-9cfb-47e2-923f-27aead50e7f3" containerName="revoke-keys" Jul 14 17:53:57.328325 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:57.328125 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a59f414-9cfb-47e2-923f-27aead50e7f3" containerName="revoke-keys" Jul 14 17:53:57.328325 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:57.328131 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a59f414-9cfb-47e2-923f-27aead50e7f3" containerName="revoke-keys" Jul 14 17:53:57.328325 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:57.328197 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a59f414-9cfb-47e2-923f-27aead50e7f3" containerName="revoke-keys" Jul 14 17:53:57.328325 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:57.328205 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a59f414-9cfb-47e2-923f-27aead50e7f3" containerName="revoke-keys" Jul 14 17:53:57.332316 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:57.332297 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-06d94c77-v4pk5" Jul 14 17:53:57.335107 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:57.335089 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9rnpk\"" Jul 14 17:53:57.339162 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:57.339141 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-06d94c77-v4pk5"] Jul 14 17:53:57.437142 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:57.437107 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b65a60c-9c51-4f91-aa0c-dda907bacd0a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-06d94c77-v4pk5\" (UID: \"6b65a60c-9c51-4f91-aa0c-dda907bacd0a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-06d94c77-v4pk5" Jul 14 17:53:57.538425 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:57.538389 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b65a60c-9c51-4f91-aa0c-dda907bacd0a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-06d94c77-v4pk5\" (UID: \"6b65a60c-9c51-4f91-aa0c-dda907bacd0a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-06d94c77-v4pk5" Jul 14 17:53:57.539013 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:57.538993 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b65a60c-9c51-4f91-aa0c-dda907bacd0a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-06d94c77-v4pk5\" (UID: \"6b65a60c-9c51-4f91-aa0c-dda907bacd0a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-06d94c77-v4pk5" Jul 14 17:53:57.644303 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:57.644266 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-06d94c77-v4pk5" Jul 14 17:53:57.980497 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:57.980414 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-06d94c77-v4pk5"] Jul 14 17:53:57.983648 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:53:57.983611 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b65a60c_9c51_4f91_aa0c_dda907bacd0a.slice/crio-d2ae7d2fdeb27a8b60c91fcdfccc6b4c6f91a292c6e99a2f27e00357d5b22cf2 WatchSource:0}: Error finding container d2ae7d2fdeb27a8b60c91fcdfccc6b4c6f91a292c6e99a2f27e00357d5b22cf2: Status 404 returned error can't find the container with id d2ae7d2fdeb27a8b60c91fcdfccc6b4c6f91a292c6e99a2f27e00357d5b22cf2 Jul 14 17:53:58.723784 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:58.723747 2554 generic.go:356] "Generic (PLEG): container finished" podID="6b65a60c-9c51-4f91-aa0c-dda907bacd0a" containerID="50177123688a2fa84d5c7b6ae3de95e0b0ea40edc96477e8254495cf2ece91ab" exitCode=0 Jul 14 17:53:58.724129 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:58.723832 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-06d94c77-v4pk5" event={"ID":"6b65a60c-9c51-4f91-aa0c-dda907bacd0a","Type":"ContainerDied","Data":"50177123688a2fa84d5c7b6ae3de95e0b0ea40edc96477e8254495cf2ece91ab"} Jul 14 17:53:58.724129 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:58.723881 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-06d94c77-v4pk5" event={"ID":"6b65a60c-9c51-4f91-aa0c-dda907bacd0a","Type":"ContainerStarted","Data":"d2ae7d2fdeb27a8b60c91fcdfccc6b4c6f91a292c6e99a2f27e00357d5b22cf2"} Jul 14 17:53:59.850937 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:59.850914 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-06d94c77-v4pk5" Jul 14 17:53:59.959613 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:59.959580 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b65a60c-9c51-4f91-aa0c-dda907bacd0a-maas-api-service-ca\") pod \"6b65a60c-9c51-4f91-aa0c-dda907bacd0a\" (UID: \"6b65a60c-9c51-4f91-aa0c-dda907bacd0a\") " Jul 14 17:53:59.959934 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:53:59.959913 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b65a60c-9c51-4f91-aa0c-dda907bacd0a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6b65a60c-9c51-4f91-aa0c-dda907bacd0a" (UID: "6b65a60c-9c51-4f91-aa0c-dda907bacd0a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:54:00.060273 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:00.060173 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b65a60c-9c51-4f91-aa0c-dda907bacd0a-maas-api-service-ca\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:54:00.735592 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:00.735509 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-06d94c77-v4pk5" event={"ID":"6b65a60c-9c51-4f91-aa0c-dda907bacd0a","Type":"ContainerDied","Data":"d2ae7d2fdeb27a8b60c91fcdfccc6b4c6f91a292c6e99a2f27e00357d5b22cf2"} Jul 14 17:54:00.735592 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:00.735539 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-06d94c77-v4pk5" Jul 14 17:54:00.735592 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:00.735561 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d2ae7d2fdeb27a8b60c91fcdfccc6b4c6f91a292c6e99a2f27e00357d5b22cf2" Jul 14 17:54:07.761260 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:07.761193 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-06d94c77-5b4978494b-tmjw8"] Jul 14 17:54:07.761635 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:07.761486 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-06d94c77-5b4978494b-tmjw8" podUID="442fcfc8-9d12-4193-8c8a-faeb59304021" containerName="maas-api" containerID="cri-o://2b57a049352e7bb66c2c67e10756a3ae2279ba38e572771eadd0d613ccc0d206" gracePeriod=30 Jul 14 17:54:08.003962 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:08.003939 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-06d94c77-5b4978494b-tmjw8" Jul 14 17:54:08.127036 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:08.127006 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxlc2\" (UniqueName: \"kubernetes.io/projected/442fcfc8-9d12-4193-8c8a-faeb59304021-kube-api-access-lxlc2\") pod \"442fcfc8-9d12-4193-8c8a-faeb59304021\" (UID: \"442fcfc8-9d12-4193-8c8a-faeb59304021\") " Jul 14 17:54:08.127190 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:08.127069 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/442fcfc8-9d12-4193-8c8a-faeb59304021-openshift-service-ca\") pod \"442fcfc8-9d12-4193-8c8a-faeb59304021\" (UID: \"442fcfc8-9d12-4193-8c8a-faeb59304021\") " Jul 14 17:54:08.127190 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:08.127096 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/442fcfc8-9d12-4193-8c8a-faeb59304021-maas-api-tls\") pod \"442fcfc8-9d12-4193-8c8a-faeb59304021\" (UID: \"442fcfc8-9d12-4193-8c8a-faeb59304021\") " Jul 14 17:54:08.127507 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:08.127484 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/442fcfc8-9d12-4193-8c8a-faeb59304021-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "442fcfc8-9d12-4193-8c8a-faeb59304021" (UID: "442fcfc8-9d12-4193-8c8a-faeb59304021"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:54:08.129243 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:08.129193 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/442fcfc8-9d12-4193-8c8a-faeb59304021-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "442fcfc8-9d12-4193-8c8a-faeb59304021" (UID: "442fcfc8-9d12-4193-8c8a-faeb59304021"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:54:08.129346 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:08.129264 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/442fcfc8-9d12-4193-8c8a-faeb59304021-kube-api-access-lxlc2" (OuterVolumeSpecName: "kube-api-access-lxlc2") pod "442fcfc8-9d12-4193-8c8a-faeb59304021" (UID: "442fcfc8-9d12-4193-8c8a-faeb59304021"). InnerVolumeSpecName "kube-api-access-lxlc2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:54:08.228625 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:08.228593 2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/442fcfc8-9d12-4193-8c8a-faeb59304021-openshift-service-ca\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:54:08.228625 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:08.228621 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/442fcfc8-9d12-4193-8c8a-faeb59304021-maas-api-tls\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:54:08.228790 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:08.228634 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxlc2\" (UniqueName: \"kubernetes.io/projected/442fcfc8-9d12-4193-8c8a-faeb59304021-kube-api-access-lxlc2\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:54:08.777842 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:08.777755 2554 generic.go:356] "Generic (PLEG): container finished" podID="442fcfc8-9d12-4193-8c8a-faeb59304021" containerID="2b57a049352e7bb66c2c67e10756a3ae2279ba38e572771eadd0d613ccc0d206" exitCode=0 Jul 14 17:54:08.777842 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:08.777818 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-06d94c77-5b4978494b-tmjw8" Jul 14 17:54:08.778306 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:08.777842 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-06d94c77-5b4978494b-tmjw8" event={"ID":"442fcfc8-9d12-4193-8c8a-faeb59304021","Type":"ContainerDied","Data":"2b57a049352e7bb66c2c67e10756a3ae2279ba38e572771eadd0d613ccc0d206"} Jul 14 17:54:08.778306 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:08.777900 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-06d94c77-5b4978494b-tmjw8" event={"ID":"442fcfc8-9d12-4193-8c8a-faeb59304021","Type":"ContainerDied","Data":"8ba0b373d55c2b1486b40af90f3f08ec1e7460680679044b527b794df6841202"} Jul 14 17:54:08.778306 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:08.777929 2554 scope.go:117] "RemoveContainer" containerID="2b57a049352e7bb66c2c67e10756a3ae2279ba38e572771eadd0d613ccc0d206" Jul 14 17:54:08.786427 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:08.786411 2554 scope.go:117] "RemoveContainer" containerID="2b57a049352e7bb66c2c67e10756a3ae2279ba38e572771eadd0d613ccc0d206" Jul 14 17:54:08.786733 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:54:08.786703 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b57a049352e7bb66c2c67e10756a3ae2279ba38e572771eadd0d613ccc0d206\": container with ID starting with 2b57a049352e7bb66c2c67e10756a3ae2279ba38e572771eadd0d613ccc0d206 not found: ID does not exist" containerID="2b57a049352e7bb66c2c67e10756a3ae2279ba38e572771eadd0d613ccc0d206" Jul 14 17:54:08.786797 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:08.786744 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b57a049352e7bb66c2c67e10756a3ae2279ba38e572771eadd0d613ccc0d206"} err="failed to get container status \"2b57a049352e7bb66c2c67e10756a3ae2279ba38e572771eadd0d613ccc0d206\": rpc error: code = NotFound desc = could not find container \"2b57a049352e7bb66c2c67e10756a3ae2279ba38e572771eadd0d613ccc0d206\": container with ID starting with 2b57a049352e7bb66c2c67e10756a3ae2279ba38e572771eadd0d613ccc0d206 not found: ID does not exist" Jul 14 17:54:08.798841 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:08.798818 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-06d94c77-5b4978494b-tmjw8"] Jul 14 17:54:08.804310 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:08.804288 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-06d94c77-5b4978494b-tmjw8"] Jul 14 17:54:10.387963 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:10.387932 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="442fcfc8-9d12-4193-8c8a-faeb59304021" path="/var/lib/kubelet/pods/442fcfc8-9d12-4193-8c8a-faeb59304021/volumes" Jul 14 17:54:10.456781 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:10.456747 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:54:24.656974 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:24.656936 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7d767796-54985f7c88-dlz86"] Jul 14 17:54:24.657392 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:24.657301 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="442fcfc8-9d12-4193-8c8a-faeb59304021" containerName="maas-api" Jul 14 17:54:24.657392 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:24.657312 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="442fcfc8-9d12-4193-8c8a-faeb59304021" containerName="maas-api" Jul 14 17:54:24.657392 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:24.657322 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b65a60c-9c51-4f91-aa0c-dda907bacd0a" containerName="revoke-keys" Jul 14 17:54:24.657392 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:24.657327 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b65a60c-9c51-4f91-aa0c-dda907bacd0a" containerName="revoke-keys" Jul 14 17:54:24.657392 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:24.657381 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="442fcfc8-9d12-4193-8c8a-faeb59304021" containerName="maas-api" Jul 14 17:54:24.657392 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:24.657388 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b65a60c-9c51-4f91-aa0c-dda907bacd0a" containerName="revoke-keys" Jul 14 17:54:24.657392 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:24.657396 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a59f414-9cfb-47e2-923f-27aead50e7f3" containerName="revoke-keys" Jul 14 17:54:24.660460 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:24.660437 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7d767796-54985f7c88-dlz86" Jul 14 17:54:24.663288 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:24.663267 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-ggvwm\"" Jul 14 17:54:24.663423 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:24.663333 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 17:54:24.664192 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:24.664174 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-7d767796-serving-cert\"" Jul 14 17:54:24.664305 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:24.664234 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 14 17:54:24.667297 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:24.667137 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frqvz\" (UniqueName: \"kubernetes.io/projected/86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8-kube-api-access-frqvz\") pod \"maas-api-e2e-migrate-7d767796-54985f7c88-dlz86\" (UID: \"86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7d767796-54985f7c88-dlz86" Jul 14 17:54:24.667297 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:24.667268 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8-openshift-service-ca\") pod \"maas-api-e2e-migrate-7d767796-54985f7c88-dlz86\" (UID: \"86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7d767796-54985f7c88-dlz86" Jul 14 17:54:24.667430 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:24.667341 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8-maas-api-tls\") pod \"maas-api-e2e-migrate-7d767796-54985f7c88-dlz86\" (UID: \"86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7d767796-54985f7c88-dlz86" Jul 14 17:54:24.677595 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:24.677572 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7d767796-54985f7c88-dlz86"] Jul 14 17:54:24.768731 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:24.768693 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8-openshift-service-ca\") pod \"maas-api-e2e-migrate-7d767796-54985f7c88-dlz86\" (UID: \"86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7d767796-54985f7c88-dlz86" Jul 14 17:54:24.768929 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:24.768749 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8-maas-api-tls\") pod \"maas-api-e2e-migrate-7d767796-54985f7c88-dlz86\" (UID: \"86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7d767796-54985f7c88-dlz86" Jul 14 17:54:24.768929 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:54:24.768857 2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-7d767796-serving-cert: secret "maas-api-e2e-migrate-7d767796-serving-cert" not found Jul 14 17:54:24.768929 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:24.768907 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frqvz\" (UniqueName: \"kubernetes.io/projected/86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8-kube-api-access-frqvz\") pod \"maas-api-e2e-migrate-7d767796-54985f7c88-dlz86\" (UID: \"86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7d767796-54985f7c88-dlz86" Jul 14 17:54:24.769106 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:54:24.768935 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8-maas-api-tls podName:86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8 nodeName:}" failed. No retries permitted until 2026-07-14 17:54:25.268918775 +0000 UTC m=+1709.410167456 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8-maas-api-tls") pod "maas-api-e2e-migrate-7d767796-54985f7c88-dlz86" (UID: "86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8") : secret "maas-api-e2e-migrate-7d767796-serving-cert" not found Jul 14 17:54:24.769452 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:24.769430 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8-openshift-service-ca\") pod \"maas-api-e2e-migrate-7d767796-54985f7c88-dlz86\" (UID: \"86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7d767796-54985f7c88-dlz86" Jul 14 17:54:24.780167 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:24.780137 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frqvz\" (UniqueName: \"kubernetes.io/projected/86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8-kube-api-access-frqvz\") pod \"maas-api-e2e-migrate-7d767796-54985f7c88-dlz86\" (UID: \"86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7d767796-54985f7c88-dlz86" Jul 14 17:54:25.272582 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:25.272552 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8-maas-api-tls\") pod \"maas-api-e2e-migrate-7d767796-54985f7c88-dlz86\" (UID: \"86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7d767796-54985f7c88-dlz86" Jul 14 17:54:25.274879 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:25.274855 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8-maas-api-tls\") pod \"maas-api-e2e-migrate-7d767796-54985f7c88-dlz86\" (UID: \"86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7d767796-54985f7c88-dlz86" Jul 14 17:54:25.571269 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:25.571182 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7d767796-54985f7c88-dlz86" Jul 14 17:54:25.714292 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:25.714258 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7d767796-54985f7c88-dlz86"] Jul 14 17:54:25.715964 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:54:25.715935 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86da5fb5_1e38_4b3a_8d55_ac0351a8d9b8.slice/crio-2d00c7a443269cab7c7ff3cd107665e9b33d8ab9f05658f65079a960bfed4f48 WatchSource:0}: Error finding container 2d00c7a443269cab7c7ff3cd107665e9b33d8ab9f05658f65079a960bfed4f48: Status 404 returned error can't find the container with id 2d00c7a443269cab7c7ff3cd107665e9b33d8ab9f05658f65079a960bfed4f48 Jul 14 17:54:25.864173 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:25.864139 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7d767796-54985f7c88-dlz86" event={"ID":"86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8","Type":"ContainerStarted","Data":"2d00c7a443269cab7c7ff3cd107665e9b33d8ab9f05658f65079a960bfed4f48"} Jul 14 17:54:26.876969 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:26.876926 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7d767796-54985f7c88-dlz86" event={"ID":"86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8","Type":"ContainerStarted","Data":"650f7c2df606fe90f19854054d40d4b42b7ae67c2451f2888422da166e6bb3c9"} Jul 14 17:54:26.877368 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:26.877017 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7d767796-54985f7c88-dlz86" Jul 14 17:54:26.899631 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:26.899578 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7d767796-54985f7c88-dlz86" podStartSLOduration=2.510798121 podStartE2EDuration="2.899563659s" totalImagesPullingTime="388.765538ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:54:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:54:26.897891601 +0000 UTC m=+1711.039140306" watchObservedRunningTime="2026-07-14 17:54:26.899563659 +0000 UTC m=+1711.040812340" Jul 14 17:54:28.949578 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:28.949547 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7d767796-jlbsc"] Jul 14 17:54:28.951854 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:28.951837 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7d767796-jlbsc" Jul 14 17:54:28.954797 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:28.954774 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9rnpk\"" Jul 14 17:54:28.961776 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:28.961754 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7d767796-jlbsc"] Jul 14 17:54:29.002993 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:29.002957 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5e6f74d-ce89-4326-8322-c1b2161cdcdc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7d767796-jlbsc\" (UID: \"c5e6f74d-ce89-4326-8322-c1b2161cdcdc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7d767796-jlbsc" Jul 14 17:54:29.103601 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:29.103564 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5e6f74d-ce89-4326-8322-c1b2161cdcdc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7d767796-jlbsc\" (UID: \"c5e6f74d-ce89-4326-8322-c1b2161cdcdc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7d767796-jlbsc" Jul 14 17:54:29.104203 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:29.104182 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5e6f74d-ce89-4326-8322-c1b2161cdcdc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7d767796-jlbsc\" (UID: \"c5e6f74d-ce89-4326-8322-c1b2161cdcdc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7d767796-jlbsc" Jul 14 17:54:29.262961 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:29.262867 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7d767796-jlbsc" Jul 14 17:54:29.394739 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:29.394715 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7d767796-jlbsc"] Jul 14 17:54:29.396373 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:54:29.396351 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5e6f74d_ce89_4326_8322_c1b2161cdcdc.slice/crio-bf338e30a3dbaa5eeb6beb5e582506c60f578bb93f5aa8a57c6d3769758bd989 WatchSource:0}: Error finding container bf338e30a3dbaa5eeb6beb5e582506c60f578bb93f5aa8a57c6d3769758bd989: Status 404 returned error can't find the container with id bf338e30a3dbaa5eeb6beb5e582506c60f578bb93f5aa8a57c6d3769758bd989 Jul 14 17:54:29.897050 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:29.897012 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7d767796-jlbsc" event={"ID":"c5e6f74d-ce89-4326-8322-c1b2161cdcdc","Type":"ContainerStarted","Data":"4679c3d986c4b1499cea1ecdc4d50c15a21f0953453e2428d7e9d52575999699"} Jul 14 17:54:29.897236 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:29.897075 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7d767796-jlbsc" event={"ID":"c5e6f74d-ce89-4326-8322-c1b2161cdcdc","Type":"ContainerStarted","Data":"bf338e30a3dbaa5eeb6beb5e582506c60f578bb93f5aa8a57c6d3769758bd989"} Jul 14 17:54:29.916595 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:29.916543 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7d767796-jlbsc" podStartSLOduration=1.916524904 podStartE2EDuration="1.916524904s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:54:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:54:29.916435661 +0000 UTC m=+1714.057684417" watchObservedRunningTime="2026-07-14 17:54:29.916524904 +0000 UTC m=+1714.057773609" Jul 14 17:54:30.903659 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:30.903617 2554 generic.go:356] "Generic (PLEG): container finished" podID="c5e6f74d-ce89-4326-8322-c1b2161cdcdc" containerID="4679c3d986c4b1499cea1ecdc4d50c15a21f0953453e2428d7e9d52575999699" exitCode=7 Jul 14 17:54:30.904066 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:30.903687 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7d767796-jlbsc" event={"ID":"c5e6f74d-ce89-4326-8322-c1b2161cdcdc","Type":"ContainerDied","Data":"4679c3d986c4b1499cea1ecdc4d50c15a21f0953453e2428d7e9d52575999699"} Jul 14 17:54:30.904066 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:30.903963 2554 scope.go:117] "RemoveContainer" containerID="4679c3d986c4b1499cea1ecdc4d50c15a21f0953453e2428d7e9d52575999699" Jul 14 17:54:31.911009 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:31.910976 2554 generic.go:356] "Generic (PLEG): container finished" podID="c5e6f74d-ce89-4326-8322-c1b2161cdcdc" containerID="04ef06dda2fc6f8f919937d86bed9f93b399d8e650d52434648baa97447e8265" exitCode=7 Jul 14 17:54:31.911431 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:31.911050 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7d767796-jlbsc" event={"ID":"c5e6f74d-ce89-4326-8322-c1b2161cdcdc","Type":"ContainerDied","Data":"04ef06dda2fc6f8f919937d86bed9f93b399d8e650d52434648baa97447e8265"} Jul 14 17:54:31.911431 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:31.911116 2554 scope.go:117] "RemoveContainer" containerID="4679c3d986c4b1499cea1ecdc4d50c15a21f0953453e2428d7e9d52575999699" Jul 14 17:54:31.911431 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:31.911380 2554 scope.go:117] "RemoveContainer" containerID="04ef06dda2fc6f8f919937d86bed9f93b399d8e650d52434648baa97447e8265" Jul 14 17:54:31.911627 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:54:31.911605 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-7d767796-jlbsc_odh-ai-gateway-infra(c5e6f74d-ce89-4326-8322-c1b2161cdcdc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7d767796-jlbsc" podUID="c5e6f74d-ce89-4326-8322-c1b2161cdcdc" Jul 14 17:54:32.886410 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:32.886375 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:54:32.887729 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:32.887706 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7d767796-54985f7c88-dlz86" Jul 14 17:54:32.919535 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:32.919510 2554 scope.go:117] "RemoveContainer" containerID="04ef06dda2fc6f8f919937d86bed9f93b399d8e650d52434648baa97447e8265" Jul 14 17:54:32.919904 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:54:32.919677 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-7d767796-jlbsc_odh-ai-gateway-infra(c5e6f74d-ce89-4326-8322-c1b2161cdcdc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7d767796-jlbsc" podUID="c5e6f74d-ce89-4326-8322-c1b2161cdcdc" Jul 14 17:54:46.385657 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:46.385621 2554 scope.go:117] "RemoveContainer" containerID="04ef06dda2fc6f8f919937d86bed9f93b399d8e650d52434648baa97447e8265" Jul 14 17:54:46.994900 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:46.994870 2554 generic.go:356] "Generic (PLEG): container finished" podID="c5e6f74d-ce89-4326-8322-c1b2161cdcdc" containerID="d10588edb7bfac6672c301e3c2e6a0ee82c779c082ca00b3e975ad3d3c08cf91" exitCode=0 Jul 14 17:54:46.995025 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:46.994939 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7d767796-jlbsc" event={"ID":"c5e6f74d-ce89-4326-8322-c1b2161cdcdc","Type":"ContainerDied","Data":"d10588edb7bfac6672c301e3c2e6a0ee82c779c082ca00b3e975ad3d3c08cf91"} Jul 14 17:54:46.995025 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:46.994999 2554 scope.go:117] "RemoveContainer" containerID="04ef06dda2fc6f8f919937d86bed9f93b399d8e650d52434648baa97447e8265" Jul 14 17:54:48.023087 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:48.023038 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7d767796-jlbsc"] Jul 14 17:54:48.132677 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:48.132655 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7d767796-jlbsc" Jul 14 17:54:48.264661 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:48.264552 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5e6f74d-ce89-4326-8322-c1b2161cdcdc-maas-api-service-ca\") pod \"c5e6f74d-ce89-4326-8322-c1b2161cdcdc\" (UID: \"c5e6f74d-ce89-4326-8322-c1b2161cdcdc\") " Jul 14 17:54:48.264933 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:48.264910 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5e6f74d-ce89-4326-8322-c1b2161cdcdc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c5e6f74d-ce89-4326-8322-c1b2161cdcdc" (UID: "c5e6f74d-ce89-4326-8322-c1b2161cdcdc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:54:48.365921 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:48.365886 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5e6f74d-ce89-4326-8322-c1b2161cdcdc-maas-api-service-ca\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:54:49.009115 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:49.009080 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7d767796-jlbsc" Jul 14 17:54:49.009308 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:49.009080 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7d767796-jlbsc" event={"ID":"c5e6f74d-ce89-4326-8322-c1b2161cdcdc","Type":"ContainerDied","Data":"bf338e30a3dbaa5eeb6beb5e582506c60f578bb93f5aa8a57c6d3769758bd989"} Jul 14 17:54:49.009308 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:49.009207 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bf338e30a3dbaa5eeb6beb5e582506c60f578bb93f5aa8a57c6d3769758bd989" Jul 14 17:54:49.044030 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:49.043992 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7d767796-jlbsc"] Jul 14 17:54:49.047937 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:49.047913 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7d767796-jlbsc"] Jul 14 17:54:50.387768 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:54:50.387732 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5e6f74d-ce89-4326-8322-c1b2161cdcdc" path="/var/lib/kubelet/pods/c5e6f74d-ce89-4326-8322-c1b2161cdcdc/volumes" Jul 14 17:55:28.994347 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:28.994310 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7d767796-wkgv7"] Jul 14 17:55:28.994891 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:28.994845 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5e6f74d-ce89-4326-8322-c1b2161cdcdc" containerName="revoke-keys" Jul 14 17:55:28.994891 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:28.994865 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5e6f74d-ce89-4326-8322-c1b2161cdcdc" containerName="revoke-keys" Jul 14 17:55:28.994891 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:28.994886 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5e6f74d-ce89-4326-8322-c1b2161cdcdc" containerName="revoke-keys" Jul 14 17:55:28.994891 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:28.994894 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5e6f74d-ce89-4326-8322-c1b2161cdcdc" containerName="revoke-keys" Jul 14 17:55:28.995110 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:28.994904 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5e6f74d-ce89-4326-8322-c1b2161cdcdc" containerName="revoke-keys" Jul 14 17:55:28.995110 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:28.994912 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5e6f74d-ce89-4326-8322-c1b2161cdcdc" containerName="revoke-keys" Jul 14 17:55:28.995110 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:28.994986 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5e6f74d-ce89-4326-8322-c1b2161cdcdc" containerName="revoke-keys" Jul 14 17:55:28.995110 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:28.994999 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5e6f74d-ce89-4326-8322-c1b2161cdcdc" containerName="revoke-keys" Jul 14 17:55:28.995110 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:28.995010 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5e6f74d-ce89-4326-8322-c1b2161cdcdc" containerName="revoke-keys" Jul 14 17:55:28.996914 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:28.996894 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7d767796-wkgv7" Jul 14 17:55:28.999660 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:28.999643 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9rnpk\"" Jul 14 17:55:29.005564 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:29.005544 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7d767796-wkgv7"] Jul 14 17:55:29.103302 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:29.103267 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/07845d1e-91ff-498a-bf41-78366f429027-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7d767796-wkgv7\" (UID: \"07845d1e-91ff-498a-bf41-78366f429027\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7d767796-wkgv7" Jul 14 17:55:29.203891 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:29.203840 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/07845d1e-91ff-498a-bf41-78366f429027-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7d767796-wkgv7\" (UID: \"07845d1e-91ff-498a-bf41-78366f429027\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7d767796-wkgv7" Jul 14 17:55:29.204534 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:29.204517 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/07845d1e-91ff-498a-bf41-78366f429027-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7d767796-wkgv7\" (UID: \"07845d1e-91ff-498a-bf41-78366f429027\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7d767796-wkgv7" Jul 14 17:55:29.309022 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:29.308942 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7d767796-wkgv7" Jul 14 17:55:29.441109 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:29.441083 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7d767796-wkgv7"] Jul 14 17:55:29.443006 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:55:29.442980 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07845d1e_91ff_498a_bf41_78366f429027.slice/crio-7815a88f6d94f1dc5ef8098e54c3e4c1f1829ad7352a049edbb823c5f59d5f7a WatchSource:0}: Error finding container 7815a88f6d94f1dc5ef8098e54c3e4c1f1829ad7352a049edbb823c5f59d5f7a: Status 404 returned error can't find the container with id 7815a88f6d94f1dc5ef8098e54c3e4c1f1829ad7352a049edbb823c5f59d5f7a Jul 14 17:55:30.217688 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:30.217653 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7d767796-wkgv7" event={"ID":"07845d1e-91ff-498a-bf41-78366f429027","Type":"ContainerStarted","Data":"2499cca3acf2a58a31db0a29ce0e83dea83f5f993e924b7ba50f5f1a1a11d4f7"} Jul 14 17:55:30.218092 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:30.217704 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7d767796-wkgv7" event={"ID":"07845d1e-91ff-498a-bf41-78366f429027","Type":"ContainerStarted","Data":"7815a88f6d94f1dc5ef8098e54c3e4c1f1829ad7352a049edbb823c5f59d5f7a"} Jul 14 17:55:30.236792 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:30.236745 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7d767796-wkgv7" podStartSLOduration=2.236731109 podStartE2EDuration="2.236731109s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:55:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:55:30.234952584 +0000 UTC m=+1774.376201300" watchObservedRunningTime="2026-07-14 17:55:30.236731109 +0000 UTC m=+1774.377979813" Jul 14 17:55:31.224257 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:31.224196 2554 generic.go:356] "Generic (PLEG): container finished" podID="07845d1e-91ff-498a-bf41-78366f429027" containerID="2499cca3acf2a58a31db0a29ce0e83dea83f5f993e924b7ba50f5f1a1a11d4f7" exitCode=0 Jul 14 17:55:31.224692 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:31.224282 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7d767796-wkgv7" event={"ID":"07845d1e-91ff-498a-bf41-78366f429027","Type":"ContainerDied","Data":"2499cca3acf2a58a31db0a29ce0e83dea83f5f993e924b7ba50f5f1a1a11d4f7"} Jul 14 17:55:31.860981 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:31.860948 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:55:32.355812 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:32.355787 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7d767796-wkgv7" Jul 14 17:55:32.430307 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:32.430281 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/07845d1e-91ff-498a-bf41-78366f429027-maas-api-service-ca\") pod \"07845d1e-91ff-498a-bf41-78366f429027\" (UID: \"07845d1e-91ff-498a-bf41-78366f429027\") " Jul 14 17:55:32.430683 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:32.430652 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07845d1e-91ff-498a-bf41-78366f429027-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "07845d1e-91ff-498a-bf41-78366f429027" (UID: "07845d1e-91ff-498a-bf41-78366f429027"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:55:32.531009 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:32.530945 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/07845d1e-91ff-498a-bf41-78366f429027-maas-api-service-ca\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:55:33.235803 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:33.235777 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7d767796-wkgv7" Jul 14 17:55:33.235982 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:33.235802 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7d767796-wkgv7" event={"ID":"07845d1e-91ff-498a-bf41-78366f429027","Type":"ContainerDied","Data":"7815a88f6d94f1dc5ef8098e54c3e4c1f1829ad7352a049edbb823c5f59d5f7a"} Jul 14 17:55:33.235982 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:33.235847 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7815a88f6d94f1dc5ef8098e54c3e4c1f1829ad7352a049edbb823c5f59d5f7a" Jul 14 17:55:34.422578 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:34.422546 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7d767796-54985f7c88-dlz86"] Jul 14 17:55:34.422988 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:34.422794 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7d767796-54985f7c88-dlz86" podUID="86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8" containerName="maas-api" containerID="cri-o://650f7c2df606fe90f19854054d40d4b42b7ae67c2451f2888422da166e6bb3c9" gracePeriod=30 Jul 14 17:55:34.668235 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:34.667132 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7d767796-54985f7c88-dlz86" Jul 14 17:55:34.749088 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:34.749058 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8-openshift-service-ca\") pod \"86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8\" (UID: \"86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8\") " Jul 14 17:55:34.749261 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:34.749103 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frqvz\" (UniqueName: \"kubernetes.io/projected/86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8-kube-api-access-frqvz\") pod \"86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8\" (UID: \"86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8\") " Jul 14 17:55:34.749261 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:34.749166 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8-maas-api-tls\") pod \"86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8\" (UID: \"86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8\") " Jul 14 17:55:34.749513 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:34.749484 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8" (UID: "86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:55:34.751185 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:34.751158 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8" (UID: "86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:55:34.751295 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:34.751237 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8-kube-api-access-frqvz" (OuterVolumeSpecName: "kube-api-access-frqvz") pod "86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8" (UID: "86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8"). InnerVolumeSpecName "kube-api-access-frqvz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:55:34.850393 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:34.850349 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8-maas-api-tls\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:55:34.850393 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:34.850389 2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8-openshift-service-ca\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:55:34.850393 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:34.850403 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frqvz\" (UniqueName: \"kubernetes.io/projected/86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8-kube-api-access-frqvz\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:55:35.247445 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:35.247413 2554 generic.go:356] "Generic (PLEG): container finished" podID="86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8" containerID="650f7c2df606fe90f19854054d40d4b42b7ae67c2451f2888422da166e6bb3c9" exitCode=0 Jul 14 17:55:35.247614 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:35.247460 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7d767796-54985f7c88-dlz86" event={"ID":"86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8","Type":"ContainerDied","Data":"650f7c2df606fe90f19854054d40d4b42b7ae67c2451f2888422da166e6bb3c9"} Jul 14 17:55:35.247614 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:35.247470 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7d767796-54985f7c88-dlz86" Jul 14 17:55:35.247614 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:35.247525 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7d767796-54985f7c88-dlz86" event={"ID":"86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8","Type":"ContainerDied","Data":"2d00c7a443269cab7c7ff3cd107665e9b33d8ab9f05658f65079a960bfed4f48"} Jul 14 17:55:35.247614 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:35.247546 2554 scope.go:117] "RemoveContainer" containerID="650f7c2df606fe90f19854054d40d4b42b7ae67c2451f2888422da166e6bb3c9" Jul 14 17:55:35.256696 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:35.256677 2554 scope.go:117] "RemoveContainer" containerID="650f7c2df606fe90f19854054d40d4b42b7ae67c2451f2888422da166e6bb3c9" Jul 14 17:55:35.256935 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:55:35.256917 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"650f7c2df606fe90f19854054d40d4b42b7ae67c2451f2888422da166e6bb3c9\": container with ID starting with 650f7c2df606fe90f19854054d40d4b42b7ae67c2451f2888422da166e6bb3c9 not found: ID does not exist" containerID="650f7c2df606fe90f19854054d40d4b42b7ae67c2451f2888422da166e6bb3c9" Jul 14 17:55:35.256976 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:35.256944 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"650f7c2df606fe90f19854054d40d4b42b7ae67c2451f2888422da166e6bb3c9"} err="failed to get container status \"650f7c2df606fe90f19854054d40d4b42b7ae67c2451f2888422da166e6bb3c9\": rpc error: code = NotFound desc = could not find container \"650f7c2df606fe90f19854054d40d4b42b7ae67c2451f2888422da166e6bb3c9\": container with ID starting with 650f7c2df606fe90f19854054d40d4b42b7ae67c2451f2888422da166e6bb3c9 not found: ID does not exist" Jul 14 17:55:35.270506 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:35.270483 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7d767796-54985f7c88-dlz86"] Jul 14 17:55:35.275749 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:35.275723 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7d767796-54985f7c88-dlz86"] Jul 14 17:55:36.388707 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:36.388671 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8" path="/var/lib/kubelet/pods/86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8/volumes" Jul 14 17:55:42.356118 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:42.356083 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-efd93356-6695b4898f-zprrz"] Jul 14 17:55:42.356654 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:42.356636 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8" containerName="maas-api" Jul 14 17:55:42.356654 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:42.356654 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8" containerName="maas-api" Jul 14 17:55:42.356796 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:42.356662 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07845d1e-91ff-498a-bf41-78366f429027" containerName="revoke-keys" Jul 14 17:55:42.356796 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:42.356668 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="07845d1e-91ff-498a-bf41-78366f429027" containerName="revoke-keys" Jul 14 17:55:42.356796 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:42.356731 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="07845d1e-91ff-498a-bf41-78366f429027" containerName="revoke-keys" Jul 14 17:55:42.356796 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:42.356740 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="86da5fb5-1e38-4b3a-8d55-ac0351a8d9b8" containerName="maas-api" Jul 14 17:55:42.361006 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:42.360988 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-efd93356-6695b4898f-zprrz" Jul 14 17:55:42.363835 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:42.363816 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-efd93356-serving-cert\"" Jul 14 17:55:42.363989 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:42.363899 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 14 17:55:42.364864 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:42.364847 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 17:55:42.364929 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:42.364901 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-ggvwm\"" Jul 14 17:55:42.375232 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:42.375190 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-efd93356-6695b4898f-zprrz"] Jul 14 17:55:42.412501 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:42.412473 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d40fa94d-7e99-4267-9014-55031d59c885-openshift-service-ca\") pod \"maas-api-e2e-ait-efd93356-6695b4898f-zprrz\" (UID: \"d40fa94d-7e99-4267-9014-55031d59c885\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-efd93356-6695b4898f-zprrz" Jul 14 17:55:42.412615 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:42.412516 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d40fa94d-7e99-4267-9014-55031d59c885-maas-api-tls\") pod \"maas-api-e2e-ait-efd93356-6695b4898f-zprrz\" (UID: \"d40fa94d-7e99-4267-9014-55031d59c885\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-efd93356-6695b4898f-zprrz" Jul 14 17:55:42.412615 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:42.412541 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg9wx\" (UniqueName: \"kubernetes.io/projected/d40fa94d-7e99-4267-9014-55031d59c885-kube-api-access-xg9wx\") pod \"maas-api-e2e-ait-efd93356-6695b4898f-zprrz\" (UID: \"d40fa94d-7e99-4267-9014-55031d59c885\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-efd93356-6695b4898f-zprrz" Jul 14 17:55:42.513769 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:42.513739 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d40fa94d-7e99-4267-9014-55031d59c885-maas-api-tls\") pod \"maas-api-e2e-ait-efd93356-6695b4898f-zprrz\" (UID: \"d40fa94d-7e99-4267-9014-55031d59c885\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-efd93356-6695b4898f-zprrz" Jul 14 17:55:42.513915 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:42.513777 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xg9wx\" (UniqueName: \"kubernetes.io/projected/d40fa94d-7e99-4267-9014-55031d59c885-kube-api-access-xg9wx\") pod \"maas-api-e2e-ait-efd93356-6695b4898f-zprrz\" (UID: \"d40fa94d-7e99-4267-9014-55031d59c885\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-efd93356-6695b4898f-zprrz" Jul 14 17:55:42.513915 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:42.513853 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d40fa94d-7e99-4267-9014-55031d59c885-openshift-service-ca\") pod \"maas-api-e2e-ait-efd93356-6695b4898f-zprrz\" (UID: \"d40fa94d-7e99-4267-9014-55031d59c885\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-efd93356-6695b4898f-zprrz" Jul 14 17:55:42.513915 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:55:42.513895 2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-efd93356-serving-cert: secret "maas-api-e2e-ait-efd93356-serving-cert" not found Jul 14 17:55:42.514018 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:55:42.513983 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d40fa94d-7e99-4267-9014-55031d59c885-maas-api-tls podName:d40fa94d-7e99-4267-9014-55031d59c885 nodeName:}" failed. No retries permitted until 2026-07-14 17:55:43.013960938 +0000 UTC m=+1787.155209633 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d40fa94d-7e99-4267-9014-55031d59c885-maas-api-tls") pod "maas-api-e2e-ait-efd93356-6695b4898f-zprrz" (UID: "d40fa94d-7e99-4267-9014-55031d59c885") : secret "maas-api-e2e-ait-efd93356-serving-cert" not found Jul 14 17:55:42.514478 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:42.514461 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d40fa94d-7e99-4267-9014-55031d59c885-openshift-service-ca\") pod \"maas-api-e2e-ait-efd93356-6695b4898f-zprrz\" (UID: \"d40fa94d-7e99-4267-9014-55031d59c885\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-efd93356-6695b4898f-zprrz" Jul 14 17:55:42.526550 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:42.526521 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg9wx\" (UniqueName: \"kubernetes.io/projected/d40fa94d-7e99-4267-9014-55031d59c885-kube-api-access-xg9wx\") pod \"maas-api-e2e-ait-efd93356-6695b4898f-zprrz\" (UID: \"d40fa94d-7e99-4267-9014-55031d59c885\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-efd93356-6695b4898f-zprrz" Jul 14 17:55:43.018807 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:43.018767 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d40fa94d-7e99-4267-9014-55031d59c885-maas-api-tls\") pod \"maas-api-e2e-ait-efd93356-6695b4898f-zprrz\" (UID: \"d40fa94d-7e99-4267-9014-55031d59c885\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-efd93356-6695b4898f-zprrz" Jul 14 17:55:43.021373 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:43.021349 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d40fa94d-7e99-4267-9014-55031d59c885-maas-api-tls\") pod \"maas-api-e2e-ait-efd93356-6695b4898f-zprrz\" (UID: \"d40fa94d-7e99-4267-9014-55031d59c885\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-efd93356-6695b4898f-zprrz" Jul 14 17:55:43.270603 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:43.270516 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-efd93356-6695b4898f-zprrz" Jul 14 17:55:43.405950 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:43.405924 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-efd93356-6695b4898f-zprrz"] Jul 14 17:55:43.407654 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:55:43.407623 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd40fa94d_7e99_4267_9014_55031d59c885.slice/crio-9c8b6091d63026cb6c8ffc59d8723c4122037f4330717db6ffe217bd21d1f5b8 WatchSource:0}: Error finding container 9c8b6091d63026cb6c8ffc59d8723c4122037f4330717db6ffe217bd21d1f5b8: Status 404 returned error can't find the container with id 9c8b6091d63026cb6c8ffc59d8723c4122037f4330717db6ffe217bd21d1f5b8 Jul 14 17:55:44.303975 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:44.303941 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-efd93356-6695b4898f-zprrz" event={"ID":"d40fa94d-7e99-4267-9014-55031d59c885","Type":"ContainerStarted","Data":"9a2aaaf5ca7d287558e8656cbcc59518887d1adddd0d64f24ac40e980d3d5f5c"} Jul 14 17:55:44.304140 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:44.304096 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-efd93356-6695b4898f-zprrz" Jul 14 17:55:44.304206 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:44.304143 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-efd93356-6695b4898f-zprrz" event={"ID":"d40fa94d-7e99-4267-9014-55031d59c885","Type":"ContainerStarted","Data":"9c8b6091d63026cb6c8ffc59d8723c4122037f4330717db6ffe217bd21d1f5b8"} Jul 14 17:55:44.323968 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:44.323924 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-efd93356-6695b4898f-zprrz" podStartSLOduration=1.815317627 podStartE2EDuration="2.323907733s" totalImagesPullingTime="508.590106ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:55:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:55:44.322198631 +0000 UTC m=+1788.463447347" watchObservedRunningTime="2026-07-14 17:55:44.323907733 +0000 UTC m=+1788.465156414" Jul 14 17:55:44.952655 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:44.952619 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:55:47.376050 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:47.376015 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-efd93356-h5qcl"] Jul 14 17:55:47.379498 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:47.379481 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-efd93356-h5qcl" Jul 14 17:55:47.382206 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:47.382175 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9rnpk\"" Jul 14 17:55:47.386388 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:47.386366 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-efd93356-h5qcl"] Jul 14 17:55:47.455852 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:47.455820 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/706f4cd1-a3a3-4fea-bedd-eaf70d6fdb4b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-efd93356-h5qcl\" (UID: \"706f4cd1-a3a3-4fea-bedd-eaf70d6fdb4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-efd93356-h5qcl" Jul 14 17:55:47.556886 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:47.556844 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/706f4cd1-a3a3-4fea-bedd-eaf70d6fdb4b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-efd93356-h5qcl\" (UID: \"706f4cd1-a3a3-4fea-bedd-eaf70d6fdb4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-efd93356-h5qcl" Jul 14 17:55:47.557418 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:47.557401 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/706f4cd1-a3a3-4fea-bedd-eaf70d6fdb4b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-efd93356-h5qcl\" (UID: \"706f4cd1-a3a3-4fea-bedd-eaf70d6fdb4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-efd93356-h5qcl" Jul 14 17:55:47.691844 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:47.691773 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-efd93356-h5qcl" Jul 14 17:55:47.818990 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:47.818965 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-efd93356-h5qcl"] Jul 14 17:55:47.820438 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:55:47.820410 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod706f4cd1_a3a3_4fea_bedd_eaf70d6fdb4b.slice/crio-4517d2937d68780adbde5f24b42f1fe114b0da38518d84ec32b0ad74bdd7b12a WatchSource:0}: Error finding container 4517d2937d68780adbde5f24b42f1fe114b0da38518d84ec32b0ad74bdd7b12a: Status 404 returned error can't find the container with id 4517d2937d68780adbde5f24b42f1fe114b0da38518d84ec32b0ad74bdd7b12a Jul 14 17:55:48.326339 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:48.326243 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-efd93356-h5qcl" event={"ID":"706f4cd1-a3a3-4fea-bedd-eaf70d6fdb4b","Type":"ContainerStarted","Data":"ddb76042e9584dc891f156de51a0ffbd99198908b0eeb595d61a6fdd0851ea72"} Jul 14 17:55:48.326339 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:48.326304 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-efd93356-h5qcl" event={"ID":"706f4cd1-a3a3-4fea-bedd-eaf70d6fdb4b","Type":"ContainerStarted","Data":"4517d2937d68780adbde5f24b42f1fe114b0da38518d84ec32b0ad74bdd7b12a"} Jul 14 17:55:48.345329 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:48.345281 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-efd93356-h5qcl" podStartSLOduration=1.345265847 podStartE2EDuration="1.345265847s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:55:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:55:48.342068456 +0000 UTC m=+1792.483317161" watchObservedRunningTime="2026-07-14 17:55:48.345265847 +0000 UTC m=+1792.486514550" Jul 14 17:55:49.333064 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:49.333032 2554 generic.go:356] "Generic (PLEG): container finished" podID="706f4cd1-a3a3-4fea-bedd-eaf70d6fdb4b" containerID="ddb76042e9584dc891f156de51a0ffbd99198908b0eeb595d61a6fdd0851ea72" exitCode=7 Jul 14 17:55:49.333688 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:49.333123 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-efd93356-h5qcl" event={"ID":"706f4cd1-a3a3-4fea-bedd-eaf70d6fdb4b","Type":"ContainerDied","Data":"ddb76042e9584dc891f156de51a0ffbd99198908b0eeb595d61a6fdd0851ea72"} Jul 14 17:55:49.333688 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:49.333322 2554 scope.go:117] "RemoveContainer" containerID="ddb76042e9584dc891f156de51a0ffbd99198908b0eeb595d61a6fdd0851ea72" Jul 14 17:55:50.314707 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:50.314623 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-efd93356-6695b4898f-zprrz" Jul 14 17:55:50.340801 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:50.340763 2554 generic.go:356] "Generic (PLEG): container finished" podID="706f4cd1-a3a3-4fea-bedd-eaf70d6fdb4b" containerID="09ac4cef41f0fc47d3a457941fa63ca965cea9f4c2d10a9b9fda098deb7984b7" exitCode=7 Jul 14 17:55:50.341283 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:50.340812 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-efd93356-h5qcl" event={"ID":"706f4cd1-a3a3-4fea-bedd-eaf70d6fdb4b","Type":"ContainerDied","Data":"09ac4cef41f0fc47d3a457941fa63ca965cea9f4c2d10a9b9fda098deb7984b7"} Jul 14 17:55:50.341283 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:50.340889 2554 scope.go:117] "RemoveContainer" containerID="ddb76042e9584dc891f156de51a0ffbd99198908b0eeb595d61a6fdd0851ea72" Jul 14 17:55:50.341283 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:50.341072 2554 scope.go:117] "RemoveContainer" containerID="09ac4cef41f0fc47d3a457941fa63ca965cea9f4c2d10a9b9fda098deb7984b7" Jul 14 17:55:50.341449 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:55:50.341328 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-efd93356-h5qcl_odh-ai-gateway-infra(706f4cd1-a3a3-4fea-bedd-eaf70d6fdb4b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-efd93356-h5qcl" podUID="706f4cd1-a3a3-4fea-bedd-eaf70d6fdb4b" Jul 14 17:55:51.347853 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:51.347824 2554 scope.go:117] "RemoveContainer" containerID="09ac4cef41f0fc47d3a457941fa63ca965cea9f4c2d10a9b9fda098deb7984b7" Jul 14 17:55:51.348226 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:55:51.348005 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-efd93356-h5qcl_odh-ai-gateway-infra(706f4cd1-a3a3-4fea-bedd-eaf70d6fdb4b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-efd93356-h5qcl" podUID="706f4cd1-a3a3-4fea-bedd-eaf70d6fdb4b" Jul 14 17:55:56.399781 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:56.399752 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pk9h2_f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726/ovn-acl-logging/0.log" Jul 14 17:55:56.403558 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:55:56.403537 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pk9h2_f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726/ovn-acl-logging/0.log" Jul 14 17:56:03.382829 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:03.382798 2554 scope.go:117] "RemoveContainer" containerID="09ac4cef41f0fc47d3a457941fa63ca965cea9f4c2d10a9b9fda098deb7984b7" Jul 14 17:56:04.415599 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:04.415565 2554 generic.go:356] "Generic (PLEG): container finished" podID="706f4cd1-a3a3-4fea-bedd-eaf70d6fdb4b" containerID="c077fb6459f71e97478a7624185e5575b7f1423cf071e7b72f353691bc5e5a07" exitCode=0 Jul 14 17:56:04.416061 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:04.415637 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-efd93356-h5qcl" event={"ID":"706f4cd1-a3a3-4fea-bedd-eaf70d6fdb4b","Type":"ContainerDied","Data":"c077fb6459f71e97478a7624185e5575b7f1423cf071e7b72f353691bc5e5a07"} Jul 14 17:56:04.416061 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:04.415692 2554 scope.go:117] "RemoveContainer" containerID="09ac4cef41f0fc47d3a457941fa63ca965cea9f4c2d10a9b9fda098deb7984b7" Jul 14 17:56:05.450019 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:05.449988 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-efd93356-h5qcl"] Jul 14 17:56:05.554111 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:05.554080 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-efd93356-h5qcl" Jul 14 17:56:05.706666 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:05.706573 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/706f4cd1-a3a3-4fea-bedd-eaf70d6fdb4b-maas-api-service-ca\") pod \"706f4cd1-a3a3-4fea-bedd-eaf70d6fdb4b\" (UID: \"706f4cd1-a3a3-4fea-bedd-eaf70d6fdb4b\") " Jul 14 17:56:05.706944 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:05.706921 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/706f4cd1-a3a3-4fea-bedd-eaf70d6fdb4b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "706f4cd1-a3a3-4fea-bedd-eaf70d6fdb4b" (UID: "706f4cd1-a3a3-4fea-bedd-eaf70d6fdb4b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:56:05.807398 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:05.807362 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/706f4cd1-a3a3-4fea-bedd-eaf70d6fdb4b-maas-api-service-ca\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:56:06.428793 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:06.428769 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-efd93356-h5qcl" Jul 14 17:56:06.428941 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:06.428813 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-efd93356-h5qcl" event={"ID":"706f4cd1-a3a3-4fea-bedd-eaf70d6fdb4b","Type":"ContainerDied","Data":"4517d2937d68780adbde5f24b42f1fe114b0da38518d84ec32b0ad74bdd7b12a"} Jul 14 17:56:06.428941 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:06.428855 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4517d2937d68780adbde5f24b42f1fe114b0da38518d84ec32b0ad74bdd7b12a" Jul 14 17:56:06.460382 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:06.460353 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-efd93356-h5qcl"] Jul 14 17:56:06.467584 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:06.467563 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-efd93356-h5qcl"] Jul 14 17:56:08.387783 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:08.387747 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="706f4cd1-a3a3-4fea-bedd-eaf70d6fdb4b" path="/var/lib/kubelet/pods/706f4cd1-a3a3-4fea-bedd-eaf70d6fdb4b/volumes" Jul 14 17:56:37.419550 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:37.419514 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-efd93356-nv84v"] Jul 14 17:56:37.420015 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:37.419856 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="706f4cd1-a3a3-4fea-bedd-eaf70d6fdb4b" containerName="revoke-keys" Jul 14 17:56:37.420015 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:37.419867 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="706f4cd1-a3a3-4fea-bedd-eaf70d6fdb4b" containerName="revoke-keys" Jul 14 17:56:37.420015 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:37.419892 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="706f4cd1-a3a3-4fea-bedd-eaf70d6fdb4b" containerName="revoke-keys" Jul 14 17:56:37.420015 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:37.419898 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="706f4cd1-a3a3-4fea-bedd-eaf70d6fdb4b" containerName="revoke-keys" Jul 14 17:56:37.420015 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:37.419958 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="706f4cd1-a3a3-4fea-bedd-eaf70d6fdb4b" containerName="revoke-keys" Jul 14 17:56:37.420015 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:37.419966 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="706f4cd1-a3a3-4fea-bedd-eaf70d6fdb4b" containerName="revoke-keys" Jul 14 17:56:37.420015 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:37.419972 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="706f4cd1-a3a3-4fea-bedd-eaf70d6fdb4b" containerName="revoke-keys" Jul 14 17:56:37.423148 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:37.423132 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-efd93356-nv84v" Jul 14 17:56:37.425954 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:37.425937 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9rnpk\"" Jul 14 17:56:37.429942 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:37.429919 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-efd93356-nv84v"] Jul 14 17:56:37.471606 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:37.471565 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8aca8919-29bb-47e5-a86e-454003ad6ccf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-efd93356-nv84v\" (UID: \"8aca8919-29bb-47e5-a86e-454003ad6ccf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-efd93356-nv84v" Jul 14 17:56:37.572547 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:37.572510 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8aca8919-29bb-47e5-a86e-454003ad6ccf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-efd93356-nv84v\" (UID: \"8aca8919-29bb-47e5-a86e-454003ad6ccf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-efd93356-nv84v" Jul 14 17:56:37.573248 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:37.573206 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8aca8919-29bb-47e5-a86e-454003ad6ccf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-efd93356-nv84v\" (UID: \"8aca8919-29bb-47e5-a86e-454003ad6ccf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-efd93356-nv84v" Jul 14 17:56:37.735198 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:37.735095 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-efd93356-nv84v" Jul 14 17:56:37.866390 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:37.866365 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-efd93356-nv84v"] Jul 14 17:56:37.868261 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:56:37.868235 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8aca8919_29bb_47e5_a86e_454003ad6ccf.slice/crio-1e18f6a40139155066bcd2b9eab777eba830b671adebad4974d7a4e89847228d WatchSource:0}: Error finding container 1e18f6a40139155066bcd2b9eab777eba830b671adebad4974d7a4e89847228d: Status 404 returned error can't find the container with id 1e18f6a40139155066bcd2b9eab777eba830b671adebad4974d7a4e89847228d Jul 14 17:56:38.594815 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:38.594785 2554 generic.go:356] "Generic (PLEG): container finished" podID="8aca8919-29bb-47e5-a86e-454003ad6ccf" containerID="0799a42b0a4c0ff45f22784371e845932855688f877270840daa6c472aca4136" exitCode=0 Jul 14 17:56:38.595117 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:38.594868 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-efd93356-nv84v" event={"ID":"8aca8919-29bb-47e5-a86e-454003ad6ccf","Type":"ContainerDied","Data":"0799a42b0a4c0ff45f22784371e845932855688f877270840daa6c472aca4136"} Jul 14 17:56:38.595117 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:38.594922 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-efd93356-nv84v" event={"ID":"8aca8919-29bb-47e5-a86e-454003ad6ccf","Type":"ContainerStarted","Data":"1e18f6a40139155066bcd2b9eab777eba830b671adebad4974d7a4e89847228d"} Jul 14 17:56:39.729683 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:39.729659 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-efd93356-nv84v" Jul 14 17:56:39.791169 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:39.791142 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8aca8919-29bb-47e5-a86e-454003ad6ccf-maas-api-service-ca\") pod \"8aca8919-29bb-47e5-a86e-454003ad6ccf\" (UID: \"8aca8919-29bb-47e5-a86e-454003ad6ccf\") " Jul 14 17:56:39.791478 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:39.791456 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8aca8919-29bb-47e5-a86e-454003ad6ccf-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8aca8919-29bb-47e5-a86e-454003ad6ccf" (UID: "8aca8919-29bb-47e5-a86e-454003ad6ccf"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:56:39.892571 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:39.892498 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8aca8919-29bb-47e5-a86e-454003ad6ccf-maas-api-service-ca\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:56:40.607562 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:40.607526 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-efd93356-nv84v" Jul 14 17:56:40.607727 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:40.607566 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-efd93356-nv84v" event={"ID":"8aca8919-29bb-47e5-a86e-454003ad6ccf","Type":"ContainerDied","Data":"1e18f6a40139155066bcd2b9eab777eba830b671adebad4974d7a4e89847228d"} Jul 14 17:56:40.607727 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:40.607607 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1e18f6a40139155066bcd2b9eab777eba830b671adebad4974d7a4e89847228d" Jul 14 17:56:47.813980 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:47.813948 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-efd93356-6695b4898f-zprrz"] Jul 14 17:56:47.814537 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:47.814275 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-efd93356-6695b4898f-zprrz" podUID="d40fa94d-7e99-4267-9014-55031d59c885" containerName="maas-api" containerID="cri-o://9a2aaaf5ca7d287558e8656cbcc59518887d1adddd0d64f24ac40e980d3d5f5c" gracePeriod=30 Jul 14 17:56:48.057721 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:48.057692 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-efd93356-6695b4898f-zprrz" Jul 14 17:56:48.159725 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:48.159693 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xg9wx\" (UniqueName: \"kubernetes.io/projected/d40fa94d-7e99-4267-9014-55031d59c885-kube-api-access-xg9wx\") pod \"d40fa94d-7e99-4267-9014-55031d59c885\" (UID: \"d40fa94d-7e99-4267-9014-55031d59c885\") " Jul 14 17:56:48.159937 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:48.159735 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d40fa94d-7e99-4267-9014-55031d59c885-openshift-service-ca\") pod \"d40fa94d-7e99-4267-9014-55031d59c885\" (UID: \"d40fa94d-7e99-4267-9014-55031d59c885\") " Jul 14 17:56:48.159937 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:48.159762 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d40fa94d-7e99-4267-9014-55031d59c885-maas-api-tls\") pod \"d40fa94d-7e99-4267-9014-55031d59c885\" (UID: \"d40fa94d-7e99-4267-9014-55031d59c885\") " Jul 14 17:56:48.160079 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:48.160054 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d40fa94d-7e99-4267-9014-55031d59c885-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d40fa94d-7e99-4267-9014-55031d59c885" (UID: "d40fa94d-7e99-4267-9014-55031d59c885"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:56:48.161740 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:48.161715 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d40fa94d-7e99-4267-9014-55031d59c885-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d40fa94d-7e99-4267-9014-55031d59c885" (UID: "d40fa94d-7e99-4267-9014-55031d59c885"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:56:48.161861 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:48.161825 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d40fa94d-7e99-4267-9014-55031d59c885-kube-api-access-xg9wx" (OuterVolumeSpecName: "kube-api-access-xg9wx") pod "d40fa94d-7e99-4267-9014-55031d59c885" (UID: "d40fa94d-7e99-4267-9014-55031d59c885"). InnerVolumeSpecName "kube-api-access-xg9wx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:56:48.261334 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:48.261287 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xg9wx\" (UniqueName: \"kubernetes.io/projected/d40fa94d-7e99-4267-9014-55031d59c885-kube-api-access-xg9wx\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:56:48.261334 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:48.261327 2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d40fa94d-7e99-4267-9014-55031d59c885-openshift-service-ca\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:56:48.261334 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:48.261338 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d40fa94d-7e99-4267-9014-55031d59c885-maas-api-tls\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:56:48.657471 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:48.657436 2554 generic.go:356] "Generic (PLEG): container finished" podID="d40fa94d-7e99-4267-9014-55031d59c885" containerID="9a2aaaf5ca7d287558e8656cbcc59518887d1adddd0d64f24ac40e980d3d5f5c" exitCode=0 Jul 14 17:56:48.657633 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:48.657494 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-efd93356-6695b4898f-zprrz" Jul 14 17:56:48.657633 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:48.657523 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-efd93356-6695b4898f-zprrz" event={"ID":"d40fa94d-7e99-4267-9014-55031d59c885","Type":"ContainerDied","Data":"9a2aaaf5ca7d287558e8656cbcc59518887d1adddd0d64f24ac40e980d3d5f5c"} Jul 14 17:56:48.657633 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:48.657580 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-efd93356-6695b4898f-zprrz" event={"ID":"d40fa94d-7e99-4267-9014-55031d59c885","Type":"ContainerDied","Data":"9c8b6091d63026cb6c8ffc59d8723c4122037f4330717db6ffe217bd21d1f5b8"} Jul 14 17:56:48.657633 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:48.657602 2554 scope.go:117] "RemoveContainer" containerID="9a2aaaf5ca7d287558e8656cbcc59518887d1adddd0d64f24ac40e980d3d5f5c" Jul 14 17:56:48.666281 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:48.666262 2554 scope.go:117] "RemoveContainer" containerID="9a2aaaf5ca7d287558e8656cbcc59518887d1adddd0d64f24ac40e980d3d5f5c" Jul 14 17:56:48.666542 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:56:48.666520 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a2aaaf5ca7d287558e8656cbcc59518887d1adddd0d64f24ac40e980d3d5f5c\": container with ID starting with 9a2aaaf5ca7d287558e8656cbcc59518887d1adddd0d64f24ac40e980d3d5f5c not found: ID does not exist" containerID="9a2aaaf5ca7d287558e8656cbcc59518887d1adddd0d64f24ac40e980d3d5f5c" Jul 14 17:56:48.666608 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:48.666553 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a2aaaf5ca7d287558e8656cbcc59518887d1adddd0d64f24ac40e980d3d5f5c"} err="failed to get container status \"9a2aaaf5ca7d287558e8656cbcc59518887d1adddd0d64f24ac40e980d3d5f5c\": rpc error: code = NotFound desc = could not find container \"9a2aaaf5ca7d287558e8656cbcc59518887d1adddd0d64f24ac40e980d3d5f5c\": container with ID starting with 9a2aaaf5ca7d287558e8656cbcc59518887d1adddd0d64f24ac40e980d3d5f5c not found: ID does not exist" Jul 14 17:56:48.676138 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:48.676116 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-efd93356-6695b4898f-zprrz"] Jul 14 17:56:48.683129 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:48.683109 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-efd93356-6695b4898f-zprrz"] Jul 14 17:56:50.348873 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:50.348836 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:56:50.388106 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:56:50.388074 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d40fa94d-7e99-4267-9014-55031d59c885" path="/var/lib/kubelet/pods/d40fa94d-7e99-4267-9014-55031d59c885/volumes" Jul 14 17:57:05.158019 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:05.157985 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-eef895f6-5856dd5b59-68nph"] Jul 14 17:57:05.158569 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:05.158399 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d40fa94d-7e99-4267-9014-55031d59c885" containerName="maas-api" Jul 14 17:57:05.158569 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:05.158413 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="d40fa94d-7e99-4267-9014-55031d59c885" containerName="maas-api" Jul 14 17:57:05.158569 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:05.158423 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="706f4cd1-a3a3-4fea-bedd-eaf70d6fdb4b" containerName="revoke-keys" Jul 14 17:57:05.158569 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:05.158429 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="706f4cd1-a3a3-4fea-bedd-eaf70d6fdb4b" containerName="revoke-keys" Jul 14 17:57:05.158569 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:05.158448 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8aca8919-29bb-47e5-a86e-454003ad6ccf" containerName="revoke-keys" Jul 14 17:57:05.158569 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:05.158454 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="8aca8919-29bb-47e5-a86e-454003ad6ccf" containerName="revoke-keys" Jul 14 17:57:05.158569 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:05.158519 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="8aca8919-29bb-47e5-a86e-454003ad6ccf" containerName="revoke-keys" Jul 14 17:57:05.158569 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:05.158535 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="d40fa94d-7e99-4267-9014-55031d59c885" containerName="maas-api" Jul 14 17:57:05.161859 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:05.161835 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-eef895f6-5856dd5b59-68nph" Jul 14 17:57:05.165199 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:05.165178 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-eef895f6-serving-cert\"" Jul 14 17:57:05.165299 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:05.165189 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-ggvwm\"" Jul 14 17:57:05.165890 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:05.165873 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 14 17:57:05.165952 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:05.165874 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 17:57:05.175105 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:05.175083 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-eef895f6-5856dd5b59-68nph"] Jul 14 17:57:05.311359 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:05.311320 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65gqq\" (UniqueName: \"kubernetes.io/projected/25975b46-a3ba-42f4-a5c2-8eae6f95365e-kube-api-access-65gqq\") pod \"maas-api-e2e-derive-eef895f6-5856dd5b59-68nph\" (UID: \"25975b46-a3ba-42f4-a5c2-8eae6f95365e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-eef895f6-5856dd5b59-68nph" Jul 14 17:57:05.311534 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:05.311381 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25975b46-a3ba-42f4-a5c2-8eae6f95365e-openshift-service-ca\") pod \"maas-api-e2e-derive-eef895f6-5856dd5b59-68nph\" (UID: \"25975b46-a3ba-42f4-a5c2-8eae6f95365e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-eef895f6-5856dd5b59-68nph" Jul 14 17:57:05.311534 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:05.311419 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25975b46-a3ba-42f4-a5c2-8eae6f95365e-maas-api-tls\") pod \"maas-api-e2e-derive-eef895f6-5856dd5b59-68nph\" (UID: \"25975b46-a3ba-42f4-a5c2-8eae6f95365e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-eef895f6-5856dd5b59-68nph" Jul 14 17:57:05.412892 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:05.412797 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65gqq\" (UniqueName: \"kubernetes.io/projected/25975b46-a3ba-42f4-a5c2-8eae6f95365e-kube-api-access-65gqq\") pod \"maas-api-e2e-derive-eef895f6-5856dd5b59-68nph\" (UID: \"25975b46-a3ba-42f4-a5c2-8eae6f95365e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-eef895f6-5856dd5b59-68nph" Jul 14 17:57:05.412892 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:05.412868 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25975b46-a3ba-42f4-a5c2-8eae6f95365e-openshift-service-ca\") pod \"maas-api-e2e-derive-eef895f6-5856dd5b59-68nph\" (UID: \"25975b46-a3ba-42f4-a5c2-8eae6f95365e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-eef895f6-5856dd5b59-68nph" Jul 14 17:57:05.413126 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:05.412911 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25975b46-a3ba-42f4-a5c2-8eae6f95365e-maas-api-tls\") pod \"maas-api-e2e-derive-eef895f6-5856dd5b59-68nph\" (UID: \"25975b46-a3ba-42f4-a5c2-8eae6f95365e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-eef895f6-5856dd5b59-68nph" Jul 14 17:57:05.413592 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:05.413567 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25975b46-a3ba-42f4-a5c2-8eae6f95365e-openshift-service-ca\") pod \"maas-api-e2e-derive-eef895f6-5856dd5b59-68nph\" (UID: \"25975b46-a3ba-42f4-a5c2-8eae6f95365e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-eef895f6-5856dd5b59-68nph" Jul 14 17:57:05.415353 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:05.415336 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25975b46-a3ba-42f4-a5c2-8eae6f95365e-maas-api-tls\") pod \"maas-api-e2e-derive-eef895f6-5856dd5b59-68nph\" (UID: \"25975b46-a3ba-42f4-a5c2-8eae6f95365e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-eef895f6-5856dd5b59-68nph" Jul 14 17:57:05.421621 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:05.421595 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65gqq\" (UniqueName: \"kubernetes.io/projected/25975b46-a3ba-42f4-a5c2-8eae6f95365e-kube-api-access-65gqq\") pod \"maas-api-e2e-derive-eef895f6-5856dd5b59-68nph\" (UID: \"25975b46-a3ba-42f4-a5c2-8eae6f95365e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-eef895f6-5856dd5b59-68nph" Jul 14 17:57:05.472417 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:05.472394 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-eef895f6-5856dd5b59-68nph" Jul 14 17:57:05.632052 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:05.632018 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-eef895f6-5856dd5b59-68nph"] Jul 14 17:57:05.633462 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:57:05.633436 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25975b46_a3ba_42f4_a5c2_8eae6f95365e.slice/crio-f3386c1a12a6a3dae91ad4191fac32619e8f0dfd6205932326198adbd3abf91b WatchSource:0}: Error finding container f3386c1a12a6a3dae91ad4191fac32619e8f0dfd6205932326198adbd3abf91b: Status 404 returned error can't find the container with id f3386c1a12a6a3dae91ad4191fac32619e8f0dfd6205932326198adbd3abf91b Jul 14 17:57:05.748261 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:05.748159 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-eef895f6-5856dd5b59-68nph" event={"ID":"25975b46-a3ba-42f4-a5c2-8eae6f95365e","Type":"ContainerStarted","Data":"f3386c1a12a6a3dae91ad4191fac32619e8f0dfd6205932326198adbd3abf91b"} Jul 14 17:57:06.755181 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:06.755136 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-eef895f6-5856dd5b59-68nph" event={"ID":"25975b46-a3ba-42f4-a5c2-8eae6f95365e","Type":"ContainerStarted","Data":"cfe8ce7eaf09f9c3bc5f84ffa19b652298a9cf515298fcb0b694097ba98b9bb3"} Jul 14 17:57:06.755616 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:06.755324 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-eef895f6-5856dd5b59-68nph" Jul 14 17:57:06.787489 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:06.787443 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-eef895f6-5856dd5b59-68nph" podStartSLOduration=1.376972689 podStartE2EDuration="1.787426305s" totalImagesPullingTime="410.453616ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:57:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:57:06.786011112 +0000 UTC m=+1870.927259818" watchObservedRunningTime="2026-07-14 17:57:06.787426305 +0000 UTC m=+1870.928674987" Jul 14 17:57:09.358789 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:09.358752 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eef895f6-5hq28"] Jul 14 17:57:09.362263 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:09.362236 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eef895f6-5hq28" Jul 14 17:57:09.364883 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:09.364866 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9rnpk\"" Jul 14 17:57:09.370602 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:09.370035 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eef895f6-5hq28"] Jul 14 17:57:09.447842 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:09.447810 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fa1cdb1-dd57-4e98-a694-6a3805dd8c31-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-eef895f6-5hq28\" (UID: \"7fa1cdb1-dd57-4e98-a694-6a3805dd8c31\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eef895f6-5hq28" Jul 14 17:57:09.549126 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:09.549093 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fa1cdb1-dd57-4e98-a694-6a3805dd8c31-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-eef895f6-5hq28\" (UID: \"7fa1cdb1-dd57-4e98-a694-6a3805dd8c31\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eef895f6-5hq28" Jul 14 17:57:09.549715 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:09.549696 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fa1cdb1-dd57-4e98-a694-6a3805dd8c31-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-eef895f6-5hq28\" (UID: \"7fa1cdb1-dd57-4e98-a694-6a3805dd8c31\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eef895f6-5hq28" Jul 14 17:57:09.674349 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:09.674269 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eef895f6-5hq28" Jul 14 17:57:09.808857 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:09.808824 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eef895f6-5hq28"] Jul 14 17:57:09.811638 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:57:09.811610 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fa1cdb1_dd57_4e98_a694_6a3805dd8c31.slice/crio-d1ed9ce9211fbdfe3a9b441b6cee3d2b6e615a43dfa16b8afcd8514144303965 WatchSource:0}: Error finding container d1ed9ce9211fbdfe3a9b441b6cee3d2b6e615a43dfa16b8afcd8514144303965: Status 404 returned error can't find the container with id d1ed9ce9211fbdfe3a9b441b6cee3d2b6e615a43dfa16b8afcd8514144303965 Jul 14 17:57:10.779648 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:10.779566 2554 generic.go:356] "Generic (PLEG): container finished" podID="7fa1cdb1-dd57-4e98-a694-6a3805dd8c31" containerID="34b139852032eed6dff3ab4b29fdb07e839fdc5a5ca78599727d821222cd59ae" exitCode=7 Jul 14 17:57:10.779648 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:10.779614 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eef895f6-5hq28" event={"ID":"7fa1cdb1-dd57-4e98-a694-6a3805dd8c31","Type":"ContainerDied","Data":"34b139852032eed6dff3ab4b29fdb07e839fdc5a5ca78599727d821222cd59ae"} Jul 14 17:57:10.780059 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:10.779650 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eef895f6-5hq28" event={"ID":"7fa1cdb1-dd57-4e98-a694-6a3805dd8c31","Type":"ContainerStarted","Data":"d1ed9ce9211fbdfe3a9b441b6cee3d2b6e615a43dfa16b8afcd8514144303965"} Jul 14 17:57:10.780059 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:10.779819 2554 scope.go:117] "RemoveContainer" containerID="34b139852032eed6dff3ab4b29fdb07e839fdc5a5ca78599727d821222cd59ae" Jul 14 17:57:11.787798 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:11.787707 2554 generic.go:356] "Generic (PLEG): container finished" podID="7fa1cdb1-dd57-4e98-a694-6a3805dd8c31" containerID="b0ef5d57073294482156dfa45dc7ad0da6a2bf0d00f5724657528ecadd8cf9fd" exitCode=7 Jul 14 17:57:11.787798 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:11.787753 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eef895f6-5hq28" event={"ID":"7fa1cdb1-dd57-4e98-a694-6a3805dd8c31","Type":"ContainerDied","Data":"b0ef5d57073294482156dfa45dc7ad0da6a2bf0d00f5724657528ecadd8cf9fd"} Jul 14 17:57:11.788333 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:11.787830 2554 scope.go:117] "RemoveContainer" containerID="34b139852032eed6dff3ab4b29fdb07e839fdc5a5ca78599727d821222cd59ae" Jul 14 17:57:11.788333 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:11.788011 2554 scope.go:117] "RemoveContainer" containerID="b0ef5d57073294482156dfa45dc7ad0da6a2bf0d00f5724657528ecadd8cf9fd" Jul 14 17:57:11.788333 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:57:11.788281 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-eef895f6-5hq28_odh-ai-gateway-infra(7fa1cdb1-dd57-4e98-a694-6a3805dd8c31)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eef895f6-5hq28" podUID="7fa1cdb1-dd57-4e98-a694-6a3805dd8c31" Jul 14 17:57:12.766085 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:12.766054 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-eef895f6-5856dd5b59-68nph" Jul 14 17:57:12.795046 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:12.795020 2554 scope.go:117] "RemoveContainer" containerID="b0ef5d57073294482156dfa45dc7ad0da6a2bf0d00f5724657528ecadd8cf9fd" Jul 14 17:57:12.795651 ip-10-0-132-13 kubenswrapper[2554]: E0714 17:57:12.795171 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-eef895f6-5hq28_odh-ai-gateway-infra(7fa1cdb1-dd57-4e98-a694-6a3805dd8c31)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eef895f6-5hq28" podUID="7fa1cdb1-dd57-4e98-a694-6a3805dd8c31" Jul 14 17:57:13.548512 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:13.548475 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:57:23.382448 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:23.382416 2554 scope.go:117] "RemoveContainer" containerID="b0ef5d57073294482156dfa45dc7ad0da6a2bf0d00f5724657528ecadd8cf9fd" Jul 14 17:57:23.855388 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:23.855351 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eef895f6-5hq28" event={"ID":"7fa1cdb1-dd57-4e98-a694-6a3805dd8c31","Type":"ContainerStarted","Data":"630090e89755366275f7f73df8199a84670e662c6a31a7f5499ad1cad7f9d2eb"} Jul 14 17:57:23.876701 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:23.876638 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eef895f6-5hq28" podStartSLOduration=14.876618074 podStartE2EDuration="14.876618074s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:57:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:57:23.872380982 +0000 UTC m=+1888.013629689" watchObservedRunningTime="2026-07-14 17:57:23.876618074 +0000 UTC m=+1888.017866779" Jul 14 17:57:24.863791 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:24.863755 2554 generic.go:356] "Generic (PLEG): container finished" podID="7fa1cdb1-dd57-4e98-a694-6a3805dd8c31" containerID="630090e89755366275f7f73df8199a84670e662c6a31a7f5499ad1cad7f9d2eb" exitCode=0 Jul 14 17:57:24.864266 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:24.863828 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eef895f6-5hq28" event={"ID":"7fa1cdb1-dd57-4e98-a694-6a3805dd8c31","Type":"ContainerDied","Data":"630090e89755366275f7f73df8199a84670e662c6a31a7f5499ad1cad7f9d2eb"} Jul 14 17:57:24.864266 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:24.863895 2554 scope.go:117] "RemoveContainer" containerID="b0ef5d57073294482156dfa45dc7ad0da6a2bf0d00f5724657528ecadd8cf9fd" Jul 14 17:57:24.890777 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:24.890750 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eef895f6-5hq28"] Jul 14 17:57:25.996497 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:25.996473 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eef895f6-5hq28" Jul 14 17:57:26.083481 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:26.083451 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fa1cdb1-dd57-4e98-a694-6a3805dd8c31-maas-api-service-ca\") pod \"7fa1cdb1-dd57-4e98-a694-6a3805dd8c31\" (UID: \"7fa1cdb1-dd57-4e98-a694-6a3805dd8c31\") " Jul 14 17:57:26.083799 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:26.083776 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7fa1cdb1-dd57-4e98-a694-6a3805dd8c31-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7fa1cdb1-dd57-4e98-a694-6a3805dd8c31" (UID: "7fa1cdb1-dd57-4e98-a694-6a3805dd8c31"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:57:26.184345 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:26.184269 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fa1cdb1-dd57-4e98-a694-6a3805dd8c31-maas-api-service-ca\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:57:26.878705 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:26.878675 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eef895f6-5hq28" Jul 14 17:57:26.878881 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:26.878709 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eef895f6-5hq28" event={"ID":"7fa1cdb1-dd57-4e98-a694-6a3805dd8c31","Type":"ContainerDied","Data":"d1ed9ce9211fbdfe3a9b441b6cee3d2b6e615a43dfa16b8afcd8514144303965"} Jul 14 17:57:26.878881 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:26.878757 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d1ed9ce9211fbdfe3a9b441b6cee3d2b6e615a43dfa16b8afcd8514144303965" Jul 14 17:57:26.907639 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:26.907606 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eef895f6-5hq28"] Jul 14 17:57:26.910865 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:26.910842 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eef895f6-5hq28"] Jul 14 17:57:28.388076 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:28.388042 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7fa1cdb1-dd57-4e98-a694-6a3805dd8c31" path="/var/lib/kubelet/pods/7fa1cdb1-dd57-4e98-a694-6a3805dd8c31/volumes" Jul 14 17:57:59.398565 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:59.398535 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eef895f6-q89w5"] Jul 14 17:57:59.399035 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:59.398885 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fa1cdb1-dd57-4e98-a694-6a3805dd8c31" containerName="revoke-keys" Jul 14 17:57:59.399035 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:59.398896 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fa1cdb1-dd57-4e98-a694-6a3805dd8c31" containerName="revoke-keys" Jul 14 17:57:59.399035 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:59.398922 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fa1cdb1-dd57-4e98-a694-6a3805dd8c31" containerName="revoke-keys" Jul 14 17:57:59.399035 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:59.398931 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fa1cdb1-dd57-4e98-a694-6a3805dd8c31" containerName="revoke-keys" Jul 14 17:57:59.399035 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:59.398940 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fa1cdb1-dd57-4e98-a694-6a3805dd8c31" containerName="revoke-keys" Jul 14 17:57:59.399035 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:59.398945 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fa1cdb1-dd57-4e98-a694-6a3805dd8c31" containerName="revoke-keys" Jul 14 17:57:59.399035 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:59.398998 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fa1cdb1-dd57-4e98-a694-6a3805dd8c31" containerName="revoke-keys" Jul 14 17:57:59.399035 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:59.399012 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fa1cdb1-dd57-4e98-a694-6a3805dd8c31" containerName="revoke-keys" Jul 14 17:57:59.402041 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:59.402026 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eef895f6-q89w5" Jul 14 17:57:59.404652 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:59.404635 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9rnpk\"" Jul 14 17:57:59.409119 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:59.409096 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eef895f6-q89w5"] Jul 14 17:57:59.563869 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:59.563840 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d358b77-9bcf-4e15-8fff-5ca588992aa7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-eef895f6-q89w5\" (UID: \"4d358b77-9bcf-4e15-8fff-5ca588992aa7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eef895f6-q89w5" Jul 14 17:57:59.664959 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:59.664862 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d358b77-9bcf-4e15-8fff-5ca588992aa7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-eef895f6-q89w5\" (UID: \"4d358b77-9bcf-4e15-8fff-5ca588992aa7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eef895f6-q89w5" Jul 14 17:57:59.665559 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:59.665534 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d358b77-9bcf-4e15-8fff-5ca588992aa7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-eef895f6-q89w5\" (UID: \"4d358b77-9bcf-4e15-8fff-5ca588992aa7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eef895f6-q89w5" Jul 14 17:57:59.714549 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:59.714519 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eef895f6-q89w5" Jul 14 17:57:59.850926 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:57:59.850897 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eef895f6-q89w5"] Jul 14 17:57:59.851940 ip-10-0-132-13 kubenswrapper[2554]: W0714 17:57:59.851916 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d358b77_9bcf_4e15_8fff_5ca588992aa7.slice/crio-1b2f8061a3c68eeca52441d7ef30d8f2e716554677ea0ed1c568fe755b0e8008 WatchSource:0}: Error finding container 1b2f8061a3c68eeca52441d7ef30d8f2e716554677ea0ed1c568fe755b0e8008: Status 404 returned error can't find the container with id 1b2f8061a3c68eeca52441d7ef30d8f2e716554677ea0ed1c568fe755b0e8008 Jul 14 17:58:00.055484 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:58:00.055411 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eef895f6-q89w5" event={"ID":"4d358b77-9bcf-4e15-8fff-5ca588992aa7","Type":"ContainerStarted","Data":"1b2f8061a3c68eeca52441d7ef30d8f2e716554677ea0ed1c568fe755b0e8008"} Jul 14 17:58:01.065768 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:58:01.065731 2554 generic.go:356] "Generic (PLEG): container finished" podID="4d358b77-9bcf-4e15-8fff-5ca588992aa7" containerID="18c885636c1d49ec96f7d10af25c9ba768d611164504e881b2799575c21fbad0" exitCode=0 Jul 14 17:58:01.066144 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:58:01.065814 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eef895f6-q89w5" event={"ID":"4d358b77-9bcf-4e15-8fff-5ca588992aa7","Type":"ContainerDied","Data":"18c885636c1d49ec96f7d10af25c9ba768d611164504e881b2799575c21fbad0"} Jul 14 17:58:02.202457 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:58:02.202431 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eef895f6-q89w5" Jul 14 17:58:02.286562 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:58:02.286530 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d358b77-9bcf-4e15-8fff-5ca588992aa7-maas-api-service-ca\") pod \"4d358b77-9bcf-4e15-8fff-5ca588992aa7\" (UID: \"4d358b77-9bcf-4e15-8fff-5ca588992aa7\") " Jul 14 17:58:02.286922 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:58:02.286897 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d358b77-9bcf-4e15-8fff-5ca588992aa7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4d358b77-9bcf-4e15-8fff-5ca588992aa7" (UID: "4d358b77-9bcf-4e15-8fff-5ca588992aa7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:58:02.387834 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:58:02.387800 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d358b77-9bcf-4e15-8fff-5ca588992aa7-maas-api-service-ca\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:58:03.077266 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:58:03.077234 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eef895f6-q89w5" Jul 14 17:58:03.077444 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:58:03.077263 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eef895f6-q89w5" event={"ID":"4d358b77-9bcf-4e15-8fff-5ca588992aa7","Type":"ContainerDied","Data":"1b2f8061a3c68eeca52441d7ef30d8f2e716554677ea0ed1c568fe755b0e8008"} Jul 14 17:58:03.077444 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:58:03.077313 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1b2f8061a3c68eeca52441d7ef30d8f2e716554677ea0ed1c568fe755b0e8008" Jul 14 17:58:09.982357 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:58:09.982309 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-eef895f6-5856dd5b59-68nph"] Jul 14 17:58:09.982818 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:58:09.982599 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-eef895f6-5856dd5b59-68nph" podUID="25975b46-a3ba-42f4-a5c2-8eae6f95365e" containerName="maas-api" containerID="cri-o://cfe8ce7eaf09f9c3bc5f84ffa19b652298a9cf515298fcb0b694097ba98b9bb3" gracePeriod=30 Jul 14 17:58:10.118204 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:58:10.118173 2554 generic.go:356] "Generic (PLEG): container finished" podID="25975b46-a3ba-42f4-a5c2-8eae6f95365e" containerID="cfe8ce7eaf09f9c3bc5f84ffa19b652298a9cf515298fcb0b694097ba98b9bb3" exitCode=0 Jul 14 17:58:10.118353 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:58:10.118250 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-eef895f6-5856dd5b59-68nph" event={"ID":"25975b46-a3ba-42f4-a5c2-8eae6f95365e","Type":"ContainerDied","Data":"cfe8ce7eaf09f9c3bc5f84ffa19b652298a9cf515298fcb0b694097ba98b9bb3"} Jul 14 17:58:10.230775 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:58:10.230752 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-eef895f6-5856dd5b59-68nph" Jul 14 17:58:10.356996 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:58:10.356967 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25975b46-a3ba-42f4-a5c2-8eae6f95365e-openshift-service-ca\") pod \"25975b46-a3ba-42f4-a5c2-8eae6f95365e\" (UID: \"25975b46-a3ba-42f4-a5c2-8eae6f95365e\") " Jul 14 17:58:10.357165 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:58:10.357060 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25975b46-a3ba-42f4-a5c2-8eae6f95365e-maas-api-tls\") pod \"25975b46-a3ba-42f4-a5c2-8eae6f95365e\" (UID: \"25975b46-a3ba-42f4-a5c2-8eae6f95365e\") " Jul 14 17:58:10.357165 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:58:10.357102 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-65gqq\" (UniqueName: \"kubernetes.io/projected/25975b46-a3ba-42f4-a5c2-8eae6f95365e-kube-api-access-65gqq\") pod \"25975b46-a3ba-42f4-a5c2-8eae6f95365e\" (UID: \"25975b46-a3ba-42f4-a5c2-8eae6f95365e\") " Jul 14 17:58:10.357391 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:58:10.357364 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25975b46-a3ba-42f4-a5c2-8eae6f95365e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "25975b46-a3ba-42f4-a5c2-8eae6f95365e" (UID: "25975b46-a3ba-42f4-a5c2-8eae6f95365e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:58:10.359146 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:58:10.359128 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25975b46-a3ba-42f4-a5c2-8eae6f95365e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "25975b46-a3ba-42f4-a5c2-8eae6f95365e" (UID: "25975b46-a3ba-42f4-a5c2-8eae6f95365e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:58:10.359272 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:58:10.359179 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25975b46-a3ba-42f4-a5c2-8eae6f95365e-kube-api-access-65gqq" (OuterVolumeSpecName: "kube-api-access-65gqq") pod "25975b46-a3ba-42f4-a5c2-8eae6f95365e" (UID: "25975b46-a3ba-42f4-a5c2-8eae6f95365e"). InnerVolumeSpecName "kube-api-access-65gqq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:58:10.458364 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:58:10.458335 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25975b46-a3ba-42f4-a5c2-8eae6f95365e-maas-api-tls\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:58:10.458364 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:58:10.458361 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-65gqq\" (UniqueName: \"kubernetes.io/projected/25975b46-a3ba-42f4-a5c2-8eae6f95365e-kube-api-access-65gqq\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:58:10.458509 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:58:10.458371 2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25975b46-a3ba-42f4-a5c2-8eae6f95365e-openshift-service-ca\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 17:58:11.125229 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:58:11.125197 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-eef895f6-5856dd5b59-68nph" Jul 14 17:58:11.125660 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:58:11.125189 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-eef895f6-5856dd5b59-68nph" event={"ID":"25975b46-a3ba-42f4-a5c2-8eae6f95365e","Type":"ContainerDied","Data":"f3386c1a12a6a3dae91ad4191fac32619e8f0dfd6205932326198adbd3abf91b"} Jul 14 17:58:11.125660 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:58:11.125347 2554 scope.go:117] "RemoveContainer" containerID="cfe8ce7eaf09f9c3bc5f84ffa19b652298a9cf515298fcb0b694097ba98b9bb3" Jul 14 17:58:11.142529 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:58:11.142486 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-eef895f6-5856dd5b59-68nph"] Jul 14 17:58:11.145561 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:58:11.145539 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-eef895f6-5856dd5b59-68nph"] Jul 14 17:58:12.388116 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:58:12.388085 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25975b46-a3ba-42f4-a5c2-8eae6f95365e" path="/var/lib/kubelet/pods/25975b46-a3ba-42f4-a5c2-8eae6f95365e/volumes" Jul 14 17:58:12.752890 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:58:12.752803 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:59:49.678107 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:59:49.678071 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 17:59:56.482585 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:59:56.482555 2554 scope.go:117] "RemoveContainer" containerID="31a97fd10d2eb18b34e285f4fe881ff4b9d7a599cb13fb753642468316a34e48" Jul 14 17:59:58.760958 ip-10-0-132-13 kubenswrapper[2554]: I0714 17:59:58.760921 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:00:00.155370 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:00.155337 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734200-ss78q"] Jul 14 18:00:00.155738 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:00.155667 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25975b46-a3ba-42f4-a5c2-8eae6f95365e" containerName="maas-api" Jul 14 18:00:00.155738 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:00.155678 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="25975b46-a3ba-42f4-a5c2-8eae6f95365e" containerName="maas-api" Jul 14 18:00:00.155738 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:00.155691 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d358b77-9bcf-4e15-8fff-5ca588992aa7" containerName="revoke-keys" Jul 14 18:00:00.155738 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:00.155697 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d358b77-9bcf-4e15-8fff-5ca588992aa7" containerName="revoke-keys" Jul 14 18:00:00.155878 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:00.155751 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="25975b46-a3ba-42f4-a5c2-8eae6f95365e" containerName="maas-api" Jul 14 18:00:00.155878 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:00.155759 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d358b77-9bcf-4e15-8fff-5ca588992aa7" containerName="revoke-keys" Jul 14 18:00:00.155878 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:00.155767 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fa1cdb1-dd57-4e98-a694-6a3805dd8c31" containerName="revoke-keys" Jul 14 18:00:00.158699 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:00.158684 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734200-ss78q" Jul 14 18:00:00.162238 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:00.162202 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9rnpk\"" Jul 14 18:00:00.167873 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:00.167854 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734200-ss78q" Jul 14 18:00:00.175727 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:00.175701 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734200-ss78q"] Jul 14 18:00:00.298356 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:00.298332 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734200-ss78q"] Jul 14 18:00:00.300193 ip-10-0-132-13 kubenswrapper[2554]: W0714 18:00:00.300167 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24730953_c360_4cda_a964_4fb4a67485fb.slice/crio-4eb2bdb08b5b5022ed9a94b77d44a6f1799830cf0da1cf0115bf06da74daa1d8 WatchSource:0}: Error finding container 4eb2bdb08b5b5022ed9a94b77d44a6f1799830cf0da1cf0115bf06da74daa1d8: Status 404 returned error can't find the container with id 4eb2bdb08b5b5022ed9a94b77d44a6f1799830cf0da1cf0115bf06da74daa1d8 Jul 14 18:00:00.707461 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:00.707423 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734200-ss78q" event={"ID":"24730953-c360-4cda-a964-4fb4a67485fb","Type":"ContainerStarted","Data":"bf2010015b5ee576c41b35547688408f0a66e67c20b9c5ac777dc9835bb3a63d"} Jul 14 18:00:00.707634 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:00.707478 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734200-ss78q" event={"ID":"24730953-c360-4cda-a964-4fb4a67485fb","Type":"ContainerStarted","Data":"4eb2bdb08b5b5022ed9a94b77d44a6f1799830cf0da1cf0115bf06da74daa1d8"} Jul 14 18:00:00.725008 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:00.724915 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734200-ss78q" podStartSLOduration=0.724900398 podStartE2EDuration="724.900398ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:00:00.724609514 +0000 UTC m=+2044.865858222" watchObservedRunningTime="2026-07-14 18:00:00.724900398 +0000 UTC m=+2044.866149115" Jul 14 18:00:01.713403 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:01.713369 2554 generic.go:356] "Generic (PLEG): container finished" podID="24730953-c360-4cda-a964-4fb4a67485fb" containerID="bf2010015b5ee576c41b35547688408f0a66e67c20b9c5ac777dc9835bb3a63d" exitCode=0 Jul 14 18:00:01.713790 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:01.713467 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734200-ss78q" event={"ID":"24730953-c360-4cda-a964-4fb4a67485fb","Type":"ContainerDied","Data":"bf2010015b5ee576c41b35547688408f0a66e67c20b9c5ac777dc9835bb3a63d"} Jul 14 18:00:02.851269 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:02.851239 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734200-ss78q" Jul 14 18:00:03.724889 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:03.724865 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734200-ss78q" Jul 14 18:00:03.725061 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:03.724863 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734200-ss78q" event={"ID":"24730953-c360-4cda-a964-4fb4a67485fb","Type":"ContainerDied","Data":"4eb2bdb08b5b5022ed9a94b77d44a6f1799830cf0da1cf0115bf06da74daa1d8"} Jul 14 18:00:03.725061 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:03.724982 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4eb2bdb08b5b5022ed9a94b77d44a6f1799830cf0da1cf0115bf06da74daa1d8" Jul 14 18:00:06.924657 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:06.924624 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f92df6e4-7699896f-ln5km"] Jul 14 18:00:06.925058 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:06.924992 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24730953-c360-4cda-a964-4fb4a67485fb" containerName="cleanup" Jul 14 18:00:06.925058 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:06.925004 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="24730953-c360-4cda-a964-4fb4a67485fb" containerName="cleanup" Jul 14 18:00:06.925151 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:06.925078 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="24730953-c360-4cda-a964-4fb4a67485fb" containerName="cleanup" Jul 14 18:00:06.928035 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:06.928018 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f92df6e4-7699896f-ln5km" Jul 14 18:00:06.932316 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:06.932295 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-f92df6e4-serving-cert\"" Jul 14 18:00:06.932442 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:06.932347 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 14 18:00:06.932442 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:06.932415 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 18:00:06.933554 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:06.933534 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-ggvwm\"" Jul 14 18:00:06.959198 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:06.959164 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f92df6e4-7699896f-ln5km"] Jul 14 18:00:07.031032 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:07.030995 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7-maas-api-tls\") pod \"maas-api-e2e-mt-f92df6e4-7699896f-ln5km\" (UID: \"8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f92df6e4-7699896f-ln5km" Jul 14 18:00:07.031207 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:07.031061 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7-openshift-service-ca\") pod \"maas-api-e2e-mt-f92df6e4-7699896f-ln5km\" (UID: \"8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f92df6e4-7699896f-ln5km" Jul 14 18:00:07.031207 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:07.031111 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsb8k\" (UniqueName: \"kubernetes.io/projected/8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7-kube-api-access-hsb8k\") pod \"maas-api-e2e-mt-f92df6e4-7699896f-ln5km\" (UID: \"8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f92df6e4-7699896f-ln5km" Jul 14 18:00:07.131890 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:07.131851 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsb8k\" (UniqueName: \"kubernetes.io/projected/8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7-kube-api-access-hsb8k\") pod \"maas-api-e2e-mt-f92df6e4-7699896f-ln5km\" (UID: \"8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f92df6e4-7699896f-ln5km" Jul 14 18:00:07.132094 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:07.131903 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7-maas-api-tls\") pod \"maas-api-e2e-mt-f92df6e4-7699896f-ln5km\" (UID: \"8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f92df6e4-7699896f-ln5km" Jul 14 18:00:07.132094 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:07.131966 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7-openshift-service-ca\") pod \"maas-api-e2e-mt-f92df6e4-7699896f-ln5km\" (UID: \"8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f92df6e4-7699896f-ln5km" Jul 14 18:00:07.132561 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:07.132541 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7-openshift-service-ca\") pod \"maas-api-e2e-mt-f92df6e4-7699896f-ln5km\" (UID: \"8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f92df6e4-7699896f-ln5km" Jul 14 18:00:07.134408 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:07.134391 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7-maas-api-tls\") pod \"maas-api-e2e-mt-f92df6e4-7699896f-ln5km\" (UID: \"8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f92df6e4-7699896f-ln5km" Jul 14 18:00:07.145957 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:07.145930 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsb8k\" (UniqueName: \"kubernetes.io/projected/8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7-kube-api-access-hsb8k\") pod \"maas-api-e2e-mt-f92df6e4-7699896f-ln5km\" (UID: \"8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f92df6e4-7699896f-ln5km" Jul 14 18:00:07.237948 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:07.237854 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f92df6e4-7699896f-ln5km" Jul 14 18:00:07.395990 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:07.395964 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f92df6e4-7699896f-ln5km"] Jul 14 18:00:07.398169 ip-10-0-132-13 kubenswrapper[2554]: W0714 18:00:07.398143 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c6bc2c3_9c3d_4956_8516_6a60a46ca2e7.slice/crio-78fbb23bba8fb195e2a10df4832efed3ce7bf3408ead15f97fc37984fc90bd12 WatchSource:0}: Error finding container 78fbb23bba8fb195e2a10df4832efed3ce7bf3408ead15f97fc37984fc90bd12: Status 404 returned error can't find the container with id 78fbb23bba8fb195e2a10df4832efed3ce7bf3408ead15f97fc37984fc90bd12 Jul 14 18:00:07.399860 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:07.399843 2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 18:00:07.748904 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:07.748870 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f92df6e4-7699896f-ln5km" event={"ID":"8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7","Type":"ContainerStarted","Data":"78fbb23bba8fb195e2a10df4832efed3ce7bf3408ead15f97fc37984fc90bd12"} Jul 14 18:00:13.783158 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:13.783120 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f92df6e4-7699896f-ln5km" event={"ID":"8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7","Type":"ContainerStarted","Data":"f85918d85b0a1e492cfb20405898238ad2b2ca8511b2f3bc914c4f99a901101d"} Jul 14 18:00:13.783542 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:13.783187 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f92df6e4-7699896f-ln5km" Jul 14 18:00:13.809476 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:13.809429 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f92df6e4-7699896f-ln5km" podStartSLOduration=2.142166742 podStartE2EDuration="7.809414881s" totalImagesPullingTime="5.667248139s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:00:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:00:13.807185789 +0000 UTC m=+2057.948434493" watchObservedRunningTime="2026-07-14 18:00:13.809414881 +0000 UTC m=+2057.950663581" Jul 14 18:00:14.336868 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:14.336832 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:00:19.793147 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:19.793075 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f92df6e4-7699896f-ln5km" Jul 14 18:00:19.822158 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:19.822127 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f92df6e4-nfgxn"] Jul 14 18:00:19.825621 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:19.825606 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f92df6e4-nfgxn" Jul 14 18:00:19.833053 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:19.833032 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9rnpk\"" Jul 14 18:00:19.864776 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:19.864749 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f92df6e4-nfgxn"] Jul 14 18:00:19.924709 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:19.924681 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2dd3bc1-0dc8-46b9-905a-f143ae2a975c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f92df6e4-nfgxn\" (UID: \"b2dd3bc1-0dc8-46b9-905a-f143ae2a975c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f92df6e4-nfgxn" Jul 14 18:00:20.025425 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:20.025389 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2dd3bc1-0dc8-46b9-905a-f143ae2a975c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f92df6e4-nfgxn\" (UID: \"b2dd3bc1-0dc8-46b9-905a-f143ae2a975c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f92df6e4-nfgxn" Jul 14 18:00:20.026041 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:20.026021 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2dd3bc1-0dc8-46b9-905a-f143ae2a975c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f92df6e4-nfgxn\" (UID: \"b2dd3bc1-0dc8-46b9-905a-f143ae2a975c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f92df6e4-nfgxn" Jul 14 18:00:20.135096 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:20.135074 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f92df6e4-nfgxn" Jul 14 18:00:20.502256 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:20.502230 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f92df6e4-nfgxn"] Jul 14 18:00:20.504058 ip-10-0-132-13 kubenswrapper[2554]: W0714 18:00:20.504030 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2dd3bc1_0dc8_46b9_905a_f143ae2a975c.slice/crio-7b416e01df0f78582f4fac28b7e3e752931ca37f952ad61b757cb909f2171dd9 WatchSource:0}: Error finding container 7b416e01df0f78582f4fac28b7e3e752931ca37f952ad61b757cb909f2171dd9: Status 404 returned error can't find the container with id 7b416e01df0f78582f4fac28b7e3e752931ca37f952ad61b757cb909f2171dd9 Jul 14 18:00:20.820449 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:20.820349 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f92df6e4-nfgxn" event={"ID":"b2dd3bc1-0dc8-46b9-905a-f143ae2a975c","Type":"ContainerStarted","Data":"53df940dc224b4191599396deb2e049c1fea38f38ea4312184f1a7e103514889"} Jul 14 18:00:20.820449 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:20.820408 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f92df6e4-nfgxn" event={"ID":"b2dd3bc1-0dc8-46b9-905a-f143ae2a975c","Type":"ContainerStarted","Data":"7b416e01df0f78582f4fac28b7e3e752931ca37f952ad61b757cb909f2171dd9"} Jul 14 18:00:20.866611 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:20.866561 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f92df6e4-nfgxn" podStartSLOduration=1.8665444770000001 podStartE2EDuration="1.866544477s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:00:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:00:20.864666904 +0000 UTC m=+2065.005915608" watchObservedRunningTime="2026-07-14 18:00:20.866544477 +0000 UTC m=+2065.007793227" Jul 14 18:00:21.826791 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:21.826756 2554 generic.go:356] "Generic (PLEG): container finished" podID="b2dd3bc1-0dc8-46b9-905a-f143ae2a975c" containerID="53df940dc224b4191599396deb2e049c1fea38f38ea4312184f1a7e103514889" exitCode=0 Jul 14 18:00:21.827202 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:21.826844 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f92df6e4-nfgxn" event={"ID":"b2dd3bc1-0dc8-46b9-905a-f143ae2a975c","Type":"ContainerDied","Data":"53df940dc224b4191599396deb2e049c1fea38f38ea4312184f1a7e103514889"} Jul 14 18:00:22.972621 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:22.972593 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f92df6e4-nfgxn" Jul 14 18:00:23.046284 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:23.046246 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2dd3bc1-0dc8-46b9-905a-f143ae2a975c-maas-api-service-ca\") pod \"b2dd3bc1-0dc8-46b9-905a-f143ae2a975c\" (UID: \"b2dd3bc1-0dc8-46b9-905a-f143ae2a975c\") " Jul 14 18:00:23.046666 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:23.046642 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2dd3bc1-0dc8-46b9-905a-f143ae2a975c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b2dd3bc1-0dc8-46b9-905a-f143ae2a975c" (UID: "b2dd3bc1-0dc8-46b9-905a-f143ae2a975c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 18:00:23.147294 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:23.147265 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2dd3bc1-0dc8-46b9-905a-f143ae2a975c-maas-api-service-ca\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:00:23.838614 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:23.838579 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f92df6e4-nfgxn" event={"ID":"b2dd3bc1-0dc8-46b9-905a-f143ae2a975c","Type":"ContainerDied","Data":"7b416e01df0f78582f4fac28b7e3e752931ca37f952ad61b757cb909f2171dd9"} Jul 14 18:00:23.838815 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:23.838635 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7b416e01df0f78582f4fac28b7e3e752931ca37f952ad61b757cb909f2171dd9" Jul 14 18:00:23.838815 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:23.838591 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f92df6e4-nfgxn" Jul 14 18:00:30.560824 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:30.560786 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f92df6e4-7699896f-ln5km"] Jul 14 18:00:30.561194 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:30.560975 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f92df6e4-7699896f-ln5km" podUID="8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7" containerName="maas-api" containerID="cri-o://f85918d85b0a1e492cfb20405898238ad2b2ca8511b2f3bc914c4f99a901101d" gracePeriod=30 Jul 14 18:00:30.816225 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:30.816148 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f92df6e4-7699896f-ln5km" Jul 14 18:00:30.875855 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:30.875820 2554 generic.go:356] "Generic (PLEG): container finished" podID="8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7" containerID="f85918d85b0a1e492cfb20405898238ad2b2ca8511b2f3bc914c4f99a901101d" exitCode=0 Jul 14 18:00:30.876026 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:30.875853 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f92df6e4-7699896f-ln5km" event={"ID":"8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7","Type":"ContainerDied","Data":"f85918d85b0a1e492cfb20405898238ad2b2ca8511b2f3bc914c4f99a901101d"} Jul 14 18:00:30.876026 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:30.875866 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f92df6e4-7699896f-ln5km" Jul 14 18:00:30.876026 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:30.875942 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f92df6e4-7699896f-ln5km" event={"ID":"8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7","Type":"ContainerDied","Data":"78fbb23bba8fb195e2a10df4832efed3ce7bf3408ead15f97fc37984fc90bd12"} Jul 14 18:00:30.876026 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:30.875966 2554 scope.go:117] "RemoveContainer" containerID="f85918d85b0a1e492cfb20405898238ad2b2ca8511b2f3bc914c4f99a901101d" Jul 14 18:00:30.884605 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:30.884587 2554 scope.go:117] "RemoveContainer" containerID="f85918d85b0a1e492cfb20405898238ad2b2ca8511b2f3bc914c4f99a901101d" Jul 14 18:00:30.884863 ip-10-0-132-13 kubenswrapper[2554]: E0714 18:00:30.884847 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f85918d85b0a1e492cfb20405898238ad2b2ca8511b2f3bc914c4f99a901101d\": container with ID starting with f85918d85b0a1e492cfb20405898238ad2b2ca8511b2f3bc914c4f99a901101d not found: ID does not exist" containerID="f85918d85b0a1e492cfb20405898238ad2b2ca8511b2f3bc914c4f99a901101d" Jul 14 18:00:30.884962 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:30.884872 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f85918d85b0a1e492cfb20405898238ad2b2ca8511b2f3bc914c4f99a901101d"} err="failed to get container status \"f85918d85b0a1e492cfb20405898238ad2b2ca8511b2f3bc914c4f99a901101d\": rpc error: code = NotFound desc = could not find container \"f85918d85b0a1e492cfb20405898238ad2b2ca8511b2f3bc914c4f99a901101d\": container with ID starting with f85918d85b0a1e492cfb20405898238ad2b2ca8511b2f3bc914c4f99a901101d not found: ID does not exist" Jul 14 18:00:30.907363 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:30.907327 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7-openshift-service-ca\") pod \"8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7\" (UID: \"8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7\") " Jul 14 18:00:30.907487 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:30.907431 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7-maas-api-tls\") pod \"8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7\" (UID: \"8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7\") " Jul 14 18:00:30.907487 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:30.907459 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hsb8k\" (UniqueName: \"kubernetes.io/projected/8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7-kube-api-access-hsb8k\") pod \"8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7\" (UID: \"8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7\") " Jul 14 18:00:30.907722 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:30.907704 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7" (UID: "8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 18:00:30.909586 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:30.909562 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7" (UID: "8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 18:00:30.909641 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:30.909628 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7-kube-api-access-hsb8k" (OuterVolumeSpecName: "kube-api-access-hsb8k") pod "8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7" (UID: "8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7"). InnerVolumeSpecName "kube-api-access-hsb8k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:00:31.008997 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:31.008959 2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7-openshift-service-ca\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:00:31.008997 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:31.008988 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7-maas-api-tls\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:00:31.008997 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:31.008997 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hsb8k\" (UniqueName: \"kubernetes.io/projected/8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7-kube-api-access-hsb8k\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:00:31.199783 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:31.199743 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f92df6e4-7699896f-ln5km"] Jul 14 18:00:31.204224 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:31.204185 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f92df6e4-7699896f-ln5km"] Jul 14 18:00:32.388130 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:32.388097 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7" path="/var/lib/kubelet/pods/8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7/volumes" Jul 14 18:00:33.459452 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:33.459408 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:00:56.435605 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:56.435577 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pk9h2_f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726/ovn-acl-logging/0.log" Jul 14 18:00:56.441424 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:56.441405 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pk9h2_f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726/ovn-acl-logging/0.log" Jul 14 18:00:56.502773 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:00:56.502751 2554 scope.go:117] "RemoveContainer" containerID="d10588edb7bfac6672c301e3c2e6a0ee82c779c082ca00b3e975ad3d3c08cf91" Jul 14 18:01:09.228615 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:09.228574 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3d8a768b-7b89b6c576-6r565"] Jul 14 18:01:09.229093 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:09.229071 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2dd3bc1-0dc8-46b9-905a-f143ae2a975c" containerName="revoke-keys" Jul 14 18:01:09.229173 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:09.229097 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2dd3bc1-0dc8-46b9-905a-f143ae2a975c" containerName="revoke-keys" Jul 14 18:01:09.229173 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:09.229113 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7" containerName="maas-api" Jul 14 18:01:09.229173 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:09.229122 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7" containerName="maas-api" Jul 14 18:01:09.229359 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:09.229235 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2dd3bc1-0dc8-46b9-905a-f143ae2a975c" containerName="revoke-keys" Jul 14 18:01:09.229359 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:09.229250 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c6bc2c3-9c3d-4956-8516-6a60a46ca2e7" containerName="maas-api" Jul 14 18:01:09.232290 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:09.232269 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d8a768b-7b89b6c576-6r565" Jul 14 18:01:09.235368 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:09.235349 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-3d8a768b-serving-cert\"" Jul 14 18:01:09.235563 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:09.235547 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-ggvwm\"" Jul 14 18:01:09.240827 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:09.240811 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 14 18:01:09.240991 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:09.240979 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 18:01:09.257268 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:09.257202 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3d8a768b-7b89b6c576-6r565"] Jul 14 18:01:09.324043 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:09.324007 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab113082-5b53-4f11-97f1-929b249a3714-openshift-service-ca\") pod \"maas-api-e2e-mt-3d8a768b-7b89b6c576-6r565\" (UID: \"ab113082-5b53-4f11-97f1-929b249a3714\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d8a768b-7b89b6c576-6r565" Jul 14 18:01:09.324198 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:09.324055 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab113082-5b53-4f11-97f1-929b249a3714-maas-api-tls\") pod \"maas-api-e2e-mt-3d8a768b-7b89b6c576-6r565\" (UID: \"ab113082-5b53-4f11-97f1-929b249a3714\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d8a768b-7b89b6c576-6r565" Jul 14 18:01:09.324198 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:09.324178 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp5rq\" (UniqueName: \"kubernetes.io/projected/ab113082-5b53-4f11-97f1-929b249a3714-kube-api-access-gp5rq\") pod \"maas-api-e2e-mt-3d8a768b-7b89b6c576-6r565\" (UID: \"ab113082-5b53-4f11-97f1-929b249a3714\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d8a768b-7b89b6c576-6r565" Jul 14 18:01:09.424761 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:09.424729 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab113082-5b53-4f11-97f1-929b249a3714-openshift-service-ca\") pod \"maas-api-e2e-mt-3d8a768b-7b89b6c576-6r565\" (UID: \"ab113082-5b53-4f11-97f1-929b249a3714\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d8a768b-7b89b6c576-6r565" Jul 14 18:01:09.424917 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:09.424772 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab113082-5b53-4f11-97f1-929b249a3714-maas-api-tls\") pod \"maas-api-e2e-mt-3d8a768b-7b89b6c576-6r565\" (UID: \"ab113082-5b53-4f11-97f1-929b249a3714\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d8a768b-7b89b6c576-6r565" Jul 14 18:01:09.424917 ip-10-0-132-13 kubenswrapper[2554]: E0714 18:01:09.424875 2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-3d8a768b-serving-cert: secret "maas-api-e2e-mt-3d8a768b-serving-cert" not found Jul 14 18:01:09.424917 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:09.424893 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gp5rq\" (UniqueName: \"kubernetes.io/projected/ab113082-5b53-4f11-97f1-929b249a3714-kube-api-access-gp5rq\") pod \"maas-api-e2e-mt-3d8a768b-7b89b6c576-6r565\" (UID: \"ab113082-5b53-4f11-97f1-929b249a3714\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d8a768b-7b89b6c576-6r565" Jul 14 18:01:09.425041 ip-10-0-132-13 kubenswrapper[2554]: E0714 18:01:09.424939 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab113082-5b53-4f11-97f1-929b249a3714-maas-api-tls podName:ab113082-5b53-4f11-97f1-929b249a3714 nodeName:}" failed. No retries permitted until 2026-07-14 18:01:09.924924117 +0000 UTC m=+2114.066172799 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ab113082-5b53-4f11-97f1-929b249a3714-maas-api-tls") pod "maas-api-e2e-mt-3d8a768b-7b89b6c576-6r565" (UID: "ab113082-5b53-4f11-97f1-929b249a3714") : secret "maas-api-e2e-mt-3d8a768b-serving-cert" not found Jul 14 18:01:09.425477 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:09.425458 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab113082-5b53-4f11-97f1-929b249a3714-openshift-service-ca\") pod \"maas-api-e2e-mt-3d8a768b-7b89b6c576-6r565\" (UID: \"ab113082-5b53-4f11-97f1-929b249a3714\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d8a768b-7b89b6c576-6r565" Jul 14 18:01:09.446705 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:09.446675 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp5rq\" (UniqueName: \"kubernetes.io/projected/ab113082-5b53-4f11-97f1-929b249a3714-kube-api-access-gp5rq\") pod \"maas-api-e2e-mt-3d8a768b-7b89b6c576-6r565\" (UID: \"ab113082-5b53-4f11-97f1-929b249a3714\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d8a768b-7b89b6c576-6r565" Jul 14 18:01:09.928874 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:09.928839 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab113082-5b53-4f11-97f1-929b249a3714-maas-api-tls\") pod \"maas-api-e2e-mt-3d8a768b-7b89b6c576-6r565\" (UID: \"ab113082-5b53-4f11-97f1-929b249a3714\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d8a768b-7b89b6c576-6r565" Jul 14 18:01:09.931178 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:09.931155 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab113082-5b53-4f11-97f1-929b249a3714-maas-api-tls\") pod \"maas-api-e2e-mt-3d8a768b-7b89b6c576-6r565\" (UID: \"ab113082-5b53-4f11-97f1-929b249a3714\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d8a768b-7b89b6c576-6r565" Jul 14 18:01:10.142640 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:10.142604 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d8a768b-7b89b6c576-6r565" Jul 14 18:01:10.277858 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:10.277828 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3d8a768b-7b89b6c576-6r565"] Jul 14 18:01:10.279276 ip-10-0-132-13 kubenswrapper[2554]: W0714 18:01:10.279247 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab113082_5b53_4f11_97f1_929b249a3714.slice/crio-a90e7fd462abe03a7c7324b757ebf217fd917213d8def28ebbb9f57ae6659522 WatchSource:0}: Error finding container a90e7fd462abe03a7c7324b757ebf217fd917213d8def28ebbb9f57ae6659522: Status 404 returned error can't find the container with id a90e7fd462abe03a7c7324b757ebf217fd917213d8def28ebbb9f57ae6659522 Jul 14 18:01:11.091150 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:11.091060 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d8a768b-7b89b6c576-6r565" event={"ID":"ab113082-5b53-4f11-97f1-929b249a3714","Type":"ContainerStarted","Data":"c7c1e6545a9cac034473fdc5e55cafa2335f908c6f00293cd86adbea429c7ae3"} Jul 14 18:01:11.091150 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:11.091108 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d8a768b-7b89b6c576-6r565" event={"ID":"ab113082-5b53-4f11-97f1-929b249a3714","Type":"ContainerStarted","Data":"a90e7fd462abe03a7c7324b757ebf217fd917213d8def28ebbb9f57ae6659522"} Jul 14 18:01:11.091363 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:11.091180 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d8a768b-7b89b6c576-6r565" Jul 14 18:01:11.111580 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:11.111535 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d8a768b-7b89b6c576-6r565" podStartSLOduration=1.70742341 podStartE2EDuration="2.111522677s" totalImagesPullingTime="404.099267ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:01:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:01:11.110526009 +0000 UTC m=+2115.251774714" watchObservedRunningTime="2026-07-14 18:01:11.111522677 +0000 UTC m=+2115.252771410" Jul 14 18:01:14.255564 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:14.255529 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:01:17.102026 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:17.101996 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d8a768b-7b89b6c576-6r565" Jul 14 18:01:33.250964 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:33.250922 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:01:48.657410 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:48.657331 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:01:51.872788 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:51.872756 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3d8a768b-9cpfn"] Jul 14 18:01:51.876154 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:51.876138 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3d8a768b-9cpfn" Jul 14 18:01:51.879868 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:51.879847 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9rnpk\"" Jul 14 18:01:51.884858 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:51.884839 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3d8a768b-9cpfn"] Jul 14 18:01:51.968340 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:51.968310 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/206c6626-f5e4-47ab-8451-4f87616cbf9a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3d8a768b-9cpfn\" (UID: \"206c6626-f5e4-47ab-8451-4f87616cbf9a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3d8a768b-9cpfn" Jul 14 18:01:52.069680 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:52.069639 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/206c6626-f5e4-47ab-8451-4f87616cbf9a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3d8a768b-9cpfn\" (UID: \"206c6626-f5e4-47ab-8451-4f87616cbf9a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3d8a768b-9cpfn" Jul 14 18:01:52.070278 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:52.070259 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/206c6626-f5e4-47ab-8451-4f87616cbf9a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3d8a768b-9cpfn\" (UID: \"206c6626-f5e4-47ab-8451-4f87616cbf9a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3d8a768b-9cpfn" Jul 14 18:01:52.188151 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:52.188078 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3d8a768b-9cpfn" Jul 14 18:01:52.316741 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:52.316717 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3d8a768b-9cpfn"] Jul 14 18:01:52.317321 ip-10-0-132-13 kubenswrapper[2554]: W0714 18:01:52.317298 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod206c6626_f5e4_47ab_8451_4f87616cbf9a.slice/crio-e4590304d6e527062e5e49a8a373401bb2c59532fc21d59bd4d1d976d517c194 WatchSource:0}: Error finding container e4590304d6e527062e5e49a8a373401bb2c59532fc21d59bd4d1d976d517c194: Status 404 returned error can't find the container with id e4590304d6e527062e5e49a8a373401bb2c59532fc21d59bd4d1d976d517c194 Jul 14 18:01:53.323003 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:53.322910 2554 generic.go:356] "Generic (PLEG): container finished" podID="206c6626-f5e4-47ab-8451-4f87616cbf9a" containerID="f1030afa8c32ed3cfddcfbcde42a9013e161d0351b275d7f3124f51618520b6b" exitCode=0 Jul 14 18:01:53.323369 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:53.323001 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3d8a768b-9cpfn" event={"ID":"206c6626-f5e4-47ab-8451-4f87616cbf9a","Type":"ContainerDied","Data":"f1030afa8c32ed3cfddcfbcde42a9013e161d0351b275d7f3124f51618520b6b"} Jul 14 18:01:53.323369 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:53.323081 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3d8a768b-9cpfn" event={"ID":"206c6626-f5e4-47ab-8451-4f87616cbf9a","Type":"ContainerStarted","Data":"e4590304d6e527062e5e49a8a373401bb2c59532fc21d59bd4d1d976d517c194"} Jul 14 18:01:54.451668 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:54.451639 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3d8a768b-9cpfn" Jul 14 18:01:54.590527 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:54.590447 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/206c6626-f5e4-47ab-8451-4f87616cbf9a-maas-api-service-ca\") pod \"206c6626-f5e4-47ab-8451-4f87616cbf9a\" (UID: \"206c6626-f5e4-47ab-8451-4f87616cbf9a\") " Jul 14 18:01:54.590779 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:54.590757 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/206c6626-f5e4-47ab-8451-4f87616cbf9a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "206c6626-f5e4-47ab-8451-4f87616cbf9a" (UID: "206c6626-f5e4-47ab-8451-4f87616cbf9a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 18:01:54.691351 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:54.691322 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/206c6626-f5e4-47ab-8451-4f87616cbf9a-maas-api-service-ca\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:01:55.334793 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:55.334768 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3d8a768b-9cpfn" Jul 14 18:01:55.334793 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:55.334775 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3d8a768b-9cpfn" event={"ID":"206c6626-f5e4-47ab-8451-4f87616cbf9a","Type":"ContainerDied","Data":"e4590304d6e527062e5e49a8a373401bb2c59532fc21d59bd4d1d976d517c194"} Jul 14 18:01:55.335012 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:01:55.334823 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e4590304d6e527062e5e49a8a373401bb2c59532fc21d59bd4d1d976d517c194" Jul 14 18:02:02.306872 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:02:02.306839 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3d8a768b-7b89b6c576-6r565"] Jul 14 18:02:02.307308 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:02:02.307049 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d8a768b-7b89b6c576-6r565" podUID="ab113082-5b53-4f11-97f1-929b249a3714" containerName="maas-api" containerID="cri-o://c7c1e6545a9cac034473fdc5e55cafa2335f908c6f00293cd86adbea429c7ae3" gracePeriod=30 Jul 14 18:02:02.549433 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:02:02.549409 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d8a768b-7b89b6c576-6r565" Jul 14 18:02:02.657250 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:02:02.657194 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gp5rq\" (UniqueName: \"kubernetes.io/projected/ab113082-5b53-4f11-97f1-929b249a3714-kube-api-access-gp5rq\") pod \"ab113082-5b53-4f11-97f1-929b249a3714\" (UID: \"ab113082-5b53-4f11-97f1-929b249a3714\") " Jul 14 18:02:02.657435 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:02:02.657291 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab113082-5b53-4f11-97f1-929b249a3714-openshift-service-ca\") pod \"ab113082-5b53-4f11-97f1-929b249a3714\" (UID: \"ab113082-5b53-4f11-97f1-929b249a3714\") " Jul 14 18:02:02.657435 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:02:02.657318 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab113082-5b53-4f11-97f1-929b249a3714-maas-api-tls\") pod \"ab113082-5b53-4f11-97f1-929b249a3714\" (UID: \"ab113082-5b53-4f11-97f1-929b249a3714\") " Jul 14 18:02:02.657664 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:02:02.657631 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab113082-5b53-4f11-97f1-929b249a3714-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ab113082-5b53-4f11-97f1-929b249a3714" (UID: "ab113082-5b53-4f11-97f1-929b249a3714"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 18:02:02.659448 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:02:02.659417 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab113082-5b53-4f11-97f1-929b249a3714-kube-api-access-gp5rq" (OuterVolumeSpecName: "kube-api-access-gp5rq") pod "ab113082-5b53-4f11-97f1-929b249a3714" (UID: "ab113082-5b53-4f11-97f1-929b249a3714"). InnerVolumeSpecName "kube-api-access-gp5rq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:02:02.659553 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:02:02.659465 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab113082-5b53-4f11-97f1-929b249a3714-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ab113082-5b53-4f11-97f1-929b249a3714" (UID: "ab113082-5b53-4f11-97f1-929b249a3714"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 18:02:02.758761 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:02:02.758728 2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab113082-5b53-4f11-97f1-929b249a3714-openshift-service-ca\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:02:02.758761 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:02:02.758757 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab113082-5b53-4f11-97f1-929b249a3714-maas-api-tls\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:02:02.758761 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:02:02.758766 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gp5rq\" (UniqueName: \"kubernetes.io/projected/ab113082-5b53-4f11-97f1-929b249a3714-kube-api-access-gp5rq\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:02:03.379678 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:02:03.379639 2554 generic.go:356] "Generic (PLEG): container finished" podID="ab113082-5b53-4f11-97f1-929b249a3714" containerID="c7c1e6545a9cac034473fdc5e55cafa2335f908c6f00293cd86adbea429c7ae3" exitCode=0 Jul 14 18:02:03.380152 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:02:03.379684 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d8a768b-7b89b6c576-6r565" Jul 14 18:02:03.380152 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:02:03.379716 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d8a768b-7b89b6c576-6r565" event={"ID":"ab113082-5b53-4f11-97f1-929b249a3714","Type":"ContainerDied","Data":"c7c1e6545a9cac034473fdc5e55cafa2335f908c6f00293cd86adbea429c7ae3"} Jul 14 18:02:03.380152 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:02:03.379775 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d8a768b-7b89b6c576-6r565" event={"ID":"ab113082-5b53-4f11-97f1-929b249a3714","Type":"ContainerDied","Data":"a90e7fd462abe03a7c7324b757ebf217fd917213d8def28ebbb9f57ae6659522"} Jul 14 18:02:03.380152 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:02:03.379798 2554 scope.go:117] "RemoveContainer" containerID="c7c1e6545a9cac034473fdc5e55cafa2335f908c6f00293cd86adbea429c7ae3" Jul 14 18:02:03.389408 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:02:03.389391 2554 scope.go:117] "RemoveContainer" containerID="c7c1e6545a9cac034473fdc5e55cafa2335f908c6f00293cd86adbea429c7ae3" Jul 14 18:02:03.389694 ip-10-0-132-13 kubenswrapper[2554]: E0714 18:02:03.389673 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7c1e6545a9cac034473fdc5e55cafa2335f908c6f00293cd86adbea429c7ae3\": container with ID starting with c7c1e6545a9cac034473fdc5e55cafa2335f908c6f00293cd86adbea429c7ae3 not found: ID does not exist" containerID="c7c1e6545a9cac034473fdc5e55cafa2335f908c6f00293cd86adbea429c7ae3" Jul 14 18:02:03.389781 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:02:03.389704 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7c1e6545a9cac034473fdc5e55cafa2335f908c6f00293cd86adbea429c7ae3"} err="failed to get container status \"c7c1e6545a9cac034473fdc5e55cafa2335f908c6f00293cd86adbea429c7ae3\": rpc error: code = NotFound desc = could not find container \"c7c1e6545a9cac034473fdc5e55cafa2335f908c6f00293cd86adbea429c7ae3\": container with ID starting with c7c1e6545a9cac034473fdc5e55cafa2335f908c6f00293cd86adbea429c7ae3 not found: ID does not exist" Jul 14 18:02:03.402624 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:02:03.402599 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3d8a768b-7b89b6c576-6r565"] Jul 14 18:02:03.406735 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:02:03.406711 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3d8a768b-7b89b6c576-6r565"] Jul 14 18:02:04.387190 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:02:04.387157 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab113082-5b53-4f11-97f1-929b249a3714" path="/var/lib/kubelet/pods/ab113082-5b53-4f11-97f1-929b249a3714/volumes" Jul 14 18:02:05.755278 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:02:05.755242 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:02:56.534866 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:02:56.534833 2554 scope.go:117] "RemoveContainer" containerID="c077fb6459f71e97478a7624185e5575b7f1423cf071e7b72f353691bc5e5a07" Jul 14 18:03:13.342649 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:03:13.342611 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:03:27.348139 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:03:27.343261 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:03:56.558949 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:03:56.558913 2554 scope.go:117] "RemoveContainer" containerID="630090e89755366275f7f73df8199a84670e662c6a31a7f5499ad1cad7f9d2eb" Jul 14 18:04:07.145562 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:07.145527 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc"] Jul 14 18:04:07.146039 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:07.145873 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="206c6626-f5e4-47ab-8451-4f87616cbf9a" containerName="revoke-keys" Jul 14 18:04:07.146039 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:07.145887 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="206c6626-f5e4-47ab-8451-4f87616cbf9a" containerName="revoke-keys" Jul 14 18:04:07.146039 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:07.145901 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab113082-5b53-4f11-97f1-929b249a3714" containerName="maas-api" Jul 14 18:04:07.146039 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:07.145907 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab113082-5b53-4f11-97f1-929b249a3714" containerName="maas-api" Jul 14 18:04:07.146039 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:07.145966 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab113082-5b53-4f11-97f1-929b249a3714" containerName="maas-api" Jul 14 18:04:07.146039 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:07.145976 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="206c6626-f5e4-47ab-8451-4f87616cbf9a" containerName="revoke-keys" Jul 14 18:04:07.148672 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:07.148656 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc" Jul 14 18:04:07.151596 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:07.151577 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 14 18:04:07.151973 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:07.151957 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 18:04:07.152762 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:07.152741 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-4ef174-serving-cert\"" Jul 14 18:04:07.152880 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:07.152830 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-ggvwm\"" Jul 14 18:04:07.159730 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:07.159710 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc"] Jul 14 18:04:07.288588 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:07.288556 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6515195-58f0-4e48-b73a-a71df779dd1e-openshift-service-ca\") pod \"maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc\" (UID: \"e6515195-58f0-4e48-b73a-a71df779dd1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc" Jul 14 18:04:07.288752 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:07.288594 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6515195-58f0-4e48-b73a-a71df779dd1e-maas-api-tls\") pod \"maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc\" (UID: \"e6515195-58f0-4e48-b73a-a71df779dd1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc" Jul 14 18:04:07.288752 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:07.288716 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mw5nv\" (UniqueName: \"kubernetes.io/projected/e6515195-58f0-4e48-b73a-a71df779dd1e-kube-api-access-mw5nv\") pod \"maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc\" (UID: \"e6515195-58f0-4e48-b73a-a71df779dd1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc" Jul 14 18:04:07.389089 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:07.389058 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw5nv\" (UniqueName: \"kubernetes.io/projected/e6515195-58f0-4e48-b73a-a71df779dd1e-kube-api-access-mw5nv\") pod \"maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc\" (UID: \"e6515195-58f0-4e48-b73a-a71df779dd1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc" Jul 14 18:04:07.389263 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:07.389095 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6515195-58f0-4e48-b73a-a71df779dd1e-openshift-service-ca\") pod \"maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc\" (UID: \"e6515195-58f0-4e48-b73a-a71df779dd1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc" Jul 14 18:04:07.389263 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:07.389113 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6515195-58f0-4e48-b73a-a71df779dd1e-maas-api-tls\") pod \"maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc\" (UID: \"e6515195-58f0-4e48-b73a-a71df779dd1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc" Jul 14 18:04:07.389263 ip-10-0-132-13 kubenswrapper[2554]: E0714 18:04:07.389196 2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-4ef174-serving-cert: secret "maas-api-e2e-inf-a-4ef174-serving-cert" not found Jul 14 18:04:07.389403 ip-10-0-132-13 kubenswrapper[2554]: E0714 18:04:07.389276 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6515195-58f0-4e48-b73a-a71df779dd1e-maas-api-tls podName:e6515195-58f0-4e48-b73a-a71df779dd1e nodeName:}" failed. No retries permitted until 2026-07-14 18:04:07.889262242 +0000 UTC m=+2292.030510923 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e6515195-58f0-4e48-b73a-a71df779dd1e-maas-api-tls") pod "maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc" (UID: "e6515195-58f0-4e48-b73a-a71df779dd1e") : secret "maas-api-e2e-inf-a-4ef174-serving-cert" not found Jul 14 18:04:07.389688 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:07.389668 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6515195-58f0-4e48-b73a-a71df779dd1e-openshift-service-ca\") pod \"maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc\" (UID: \"e6515195-58f0-4e48-b73a-a71df779dd1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc" Jul 14 18:04:07.398754 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:07.398691 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mw5nv\" (UniqueName: \"kubernetes.io/projected/e6515195-58f0-4e48-b73a-a71df779dd1e-kube-api-access-mw5nv\") pod \"maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc\" (UID: \"e6515195-58f0-4e48-b73a-a71df779dd1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc" Jul 14 18:04:07.892945 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:07.892912 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6515195-58f0-4e48-b73a-a71df779dd1e-maas-api-tls\") pod \"maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc\" (UID: \"e6515195-58f0-4e48-b73a-a71df779dd1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc" Jul 14 18:04:07.895379 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:07.895356 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6515195-58f0-4e48-b73a-a71df779dd1e-maas-api-tls\") pod \"maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc\" (UID: \"e6515195-58f0-4e48-b73a-a71df779dd1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc" Jul 14 18:04:08.059471 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:08.059440 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc" Jul 14 18:04:08.186321 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:08.186288 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc"] Jul 14 18:04:08.187522 ip-10-0-132-13 kubenswrapper[2554]: W0714 18:04:08.187499 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6515195_58f0_4e48_b73a_a71df779dd1e.slice/crio-f265fbf1ff85f2ba5b95fd69929c604cd68ccbac9e274f2d0aa821cc78e66730 WatchSource:0}: Error finding container f265fbf1ff85f2ba5b95fd69929c604cd68ccbac9e274f2d0aa821cc78e66730: Status 404 returned error can't find the container with id f265fbf1ff85f2ba5b95fd69929c604cd68ccbac9e274f2d0aa821cc78e66730 Jul 14 18:04:09.053838 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:09.053749 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc" event={"ID":"e6515195-58f0-4e48-b73a-a71df779dd1e","Type":"ContainerStarted","Data":"89dea6fa38b541f405c4bc48e7cad576c11939e722ca2ce3c8ed2af5173f079f"} Jul 14 18:04:09.053838 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:09.053806 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc" event={"ID":"e6515195-58f0-4e48-b73a-a71df779dd1e","Type":"ContainerStarted","Data":"f265fbf1ff85f2ba5b95fd69929c604cd68ccbac9e274f2d0aa821cc78e66730"} Jul 14 18:04:09.054015 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:09.053955 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc" Jul 14 18:04:09.072288 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:09.072247 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc" podStartSLOduration=1.539313716 podStartE2EDuration="2.072234702s" totalImagesPullingTime="532.920986ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:04:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:04:09.070204938 +0000 UTC m=+2293.211453645" watchObservedRunningTime="2026-07-14 18:04:09.072234702 +0000 UTC m=+2293.213483414" Jul 14 18:04:11.553439 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:11.553394 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:04:15.065697 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:15.065666 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc" Jul 14 18:04:15.298504 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:15.298468 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-2d5136-76bf549667-6fxjc"] Jul 14 18:04:15.302273 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:15.302258 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2d5136-76bf549667-6fxjc" Jul 14 18:04:15.305448 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:15.305433 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-2d5136-serving-cert\"" Jul 14 18:04:15.313860 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:15.313838 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-2d5136-76bf549667-6fxjc"] Jul 14 18:04:15.459543 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:15.459509 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1920c592-bac3-4a0f-9665-9689445c36a9-maas-api-tls\") pod \"maas-api-e2e-inf-b-2d5136-76bf549667-6fxjc\" (UID: \"1920c592-bac3-4a0f-9665-9689445c36a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2d5136-76bf549667-6fxjc" Jul 14 18:04:15.459719 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:15.459550 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1920c592-bac3-4a0f-9665-9689445c36a9-openshift-service-ca\") pod \"maas-api-e2e-inf-b-2d5136-76bf549667-6fxjc\" (UID: \"1920c592-bac3-4a0f-9665-9689445c36a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2d5136-76bf549667-6fxjc" Jul 14 18:04:15.459719 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:15.459586 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgdr5\" (UniqueName: \"kubernetes.io/projected/1920c592-bac3-4a0f-9665-9689445c36a9-kube-api-access-vgdr5\") pod \"maas-api-e2e-inf-b-2d5136-76bf549667-6fxjc\" (UID: \"1920c592-bac3-4a0f-9665-9689445c36a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2d5136-76bf549667-6fxjc" Jul 14 18:04:15.560542 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:15.560500 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgdr5\" (UniqueName: \"kubernetes.io/projected/1920c592-bac3-4a0f-9665-9689445c36a9-kube-api-access-vgdr5\") pod \"maas-api-e2e-inf-b-2d5136-76bf549667-6fxjc\" (UID: \"1920c592-bac3-4a0f-9665-9689445c36a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2d5136-76bf549667-6fxjc" Jul 14 18:04:15.560728 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:15.560623 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1920c592-bac3-4a0f-9665-9689445c36a9-maas-api-tls\") pod \"maas-api-e2e-inf-b-2d5136-76bf549667-6fxjc\" (UID: \"1920c592-bac3-4a0f-9665-9689445c36a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2d5136-76bf549667-6fxjc" Jul 14 18:04:15.560728 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:15.560666 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1920c592-bac3-4a0f-9665-9689445c36a9-openshift-service-ca\") pod \"maas-api-e2e-inf-b-2d5136-76bf549667-6fxjc\" (UID: \"1920c592-bac3-4a0f-9665-9689445c36a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2d5136-76bf549667-6fxjc" Jul 14 18:04:15.561375 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:15.561354 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1920c592-bac3-4a0f-9665-9689445c36a9-openshift-service-ca\") pod \"maas-api-e2e-inf-b-2d5136-76bf549667-6fxjc\" (UID: \"1920c592-bac3-4a0f-9665-9689445c36a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2d5136-76bf549667-6fxjc" Jul 14 18:04:15.563557 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:15.563528 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1920c592-bac3-4a0f-9665-9689445c36a9-maas-api-tls\") pod \"maas-api-e2e-inf-b-2d5136-76bf549667-6fxjc\" (UID: \"1920c592-bac3-4a0f-9665-9689445c36a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2d5136-76bf549667-6fxjc" Jul 14 18:04:15.570089 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:15.570060 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgdr5\" (UniqueName: \"kubernetes.io/projected/1920c592-bac3-4a0f-9665-9689445c36a9-kube-api-access-vgdr5\") pod \"maas-api-e2e-inf-b-2d5136-76bf549667-6fxjc\" (UID: \"1920c592-bac3-4a0f-9665-9689445c36a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2d5136-76bf549667-6fxjc" Jul 14 18:04:15.613001 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:15.612973 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2d5136-76bf549667-6fxjc" Jul 14 18:04:15.743632 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:15.743605 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-2d5136-76bf549667-6fxjc"] Jul 14 18:04:15.745403 ip-10-0-132-13 kubenswrapper[2554]: W0714 18:04:15.745361 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1920c592_bac3_4a0f_9665_9689445c36a9.slice/crio-7363dfc5e94009867cc9d34df8de5a8fb42a0492381215ebe629fa2258283560 WatchSource:0}: Error finding container 7363dfc5e94009867cc9d34df8de5a8fb42a0492381215ebe629fa2258283560: Status 404 returned error can't find the container with id 7363dfc5e94009867cc9d34df8de5a8fb42a0492381215ebe629fa2258283560 Jul 14 18:04:16.091233 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:16.091190 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2d5136-76bf549667-6fxjc" event={"ID":"1920c592-bac3-4a0f-9665-9689445c36a9","Type":"ContainerStarted","Data":"7363dfc5e94009867cc9d34df8de5a8fb42a0492381215ebe629fa2258283560"} Jul 14 18:04:17.098438 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:17.098401 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2d5136-76bf549667-6fxjc" event={"ID":"1920c592-bac3-4a0f-9665-9689445c36a9","Type":"ContainerStarted","Data":"bf4d192f5e2e40d51a9cc6e88e10e0c1b52992f7026b437f59f76231a3a66440"} Jul 14 18:04:17.098850 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:17.098485 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2d5136-76bf549667-6fxjc" Jul 14 18:04:17.117474 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:17.117423 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2d5136-76bf549667-6fxjc" podStartSLOduration=1.79745766 podStartE2EDuration="2.117408211s" totalImagesPullingTime="319.950551ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:04:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:04:17.116045212 +0000 UTC m=+2301.257293918" watchObservedRunningTime="2026-07-14 18:04:17.117408211 +0000 UTC m=+2301.258656943" Jul 14 18:04:23.108946 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:23.108910 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2d5136-76bf549667-6fxjc" Jul 14 18:04:28.553971 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:28.553929 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:04:50.193272 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:50.193173 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-2d5136/test-model-2d5136-kserve-5468fdc659-d7dh8"] Jul 14 18:04:50.196812 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:50.196789 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2d5136/test-model-2d5136-kserve-5468fdc659-d7dh8" Jul 14 18:04:50.202981 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:50.202957 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-2d5136\"/\"openshift-service-ca.crt\"" Jul 14 18:04:50.203162 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:50.202977 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-2d5136\"/\"test-model-2d5136-kserve-self-signed-certs\"" Jul 14 18:04:50.203312 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:50.202977 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-2d5136\"/\"kube-root-ca.crt\"" Jul 14 18:04:50.203386 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:50.203081 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-2d5136\"/\"default-dockercfg-7b8hf\"" Jul 14 18:04:50.206553 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:50.206531 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-2d5136/test-model-2d5136-kserve-5468fdc659-d7dh8"] Jul 14 18:04:50.260314 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:50.260285 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b67c8ffe-9e33-4df2-8b8c-a7200eca9764-tmp-dir\") pod \"test-model-2d5136-kserve-5468fdc659-d7dh8\" (UID: \"b67c8ffe-9e33-4df2-8b8c-a7200eca9764\") " pod="ai-tenant-e2e-inf-b-2d5136/test-model-2d5136-kserve-5468fdc659-d7dh8" Jul 14 18:04:50.260490 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:50.260338 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b67c8ffe-9e33-4df2-8b8c-a7200eca9764-model-cache\") pod \"test-model-2d5136-kserve-5468fdc659-d7dh8\" (UID: \"b67c8ffe-9e33-4df2-8b8c-a7200eca9764\") " pod="ai-tenant-e2e-inf-b-2d5136/test-model-2d5136-kserve-5468fdc659-d7dh8" Jul 14 18:04:50.260490 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:50.260359 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b67c8ffe-9e33-4df2-8b8c-a7200eca9764-home\") pod \"test-model-2d5136-kserve-5468fdc659-d7dh8\" (UID: \"b67c8ffe-9e33-4df2-8b8c-a7200eca9764\") " pod="ai-tenant-e2e-inf-b-2d5136/test-model-2d5136-kserve-5468fdc659-d7dh8" Jul 14 18:04:50.260490 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:50.260442 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrhz7\" (UniqueName: \"kubernetes.io/projected/b67c8ffe-9e33-4df2-8b8c-a7200eca9764-kube-api-access-vrhz7\") pod \"test-model-2d5136-kserve-5468fdc659-d7dh8\" (UID: \"b67c8ffe-9e33-4df2-8b8c-a7200eca9764\") " pod="ai-tenant-e2e-inf-b-2d5136/test-model-2d5136-kserve-5468fdc659-d7dh8" Jul 14 18:04:50.260604 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:50.260487 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b67c8ffe-9e33-4df2-8b8c-a7200eca9764-tls-certs\") pod \"test-model-2d5136-kserve-5468fdc659-d7dh8\" (UID: \"b67c8ffe-9e33-4df2-8b8c-a7200eca9764\") " pod="ai-tenant-e2e-inf-b-2d5136/test-model-2d5136-kserve-5468fdc659-d7dh8" Jul 14 18:04:50.260604 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:50.260521 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b67c8ffe-9e33-4df2-8b8c-a7200eca9764-dshm\") pod \"test-model-2d5136-kserve-5468fdc659-d7dh8\" (UID: \"b67c8ffe-9e33-4df2-8b8c-a7200eca9764\") " pod="ai-tenant-e2e-inf-b-2d5136/test-model-2d5136-kserve-5468fdc659-d7dh8" Jul 14 18:04:50.361582 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:50.361550 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b67c8ffe-9e33-4df2-8b8c-a7200eca9764-model-cache\") pod \"test-model-2d5136-kserve-5468fdc659-d7dh8\" (UID: \"b67c8ffe-9e33-4df2-8b8c-a7200eca9764\") " pod="ai-tenant-e2e-inf-b-2d5136/test-model-2d5136-kserve-5468fdc659-d7dh8" Jul 14 18:04:50.361766 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:50.361594 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b67c8ffe-9e33-4df2-8b8c-a7200eca9764-home\") pod \"test-model-2d5136-kserve-5468fdc659-d7dh8\" (UID: \"b67c8ffe-9e33-4df2-8b8c-a7200eca9764\") " pod="ai-tenant-e2e-inf-b-2d5136/test-model-2d5136-kserve-5468fdc659-d7dh8" Jul 14 18:04:50.361766 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:50.361644 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrhz7\" (UniqueName: \"kubernetes.io/projected/b67c8ffe-9e33-4df2-8b8c-a7200eca9764-kube-api-access-vrhz7\") pod \"test-model-2d5136-kserve-5468fdc659-d7dh8\" (UID: \"b67c8ffe-9e33-4df2-8b8c-a7200eca9764\") " pod="ai-tenant-e2e-inf-b-2d5136/test-model-2d5136-kserve-5468fdc659-d7dh8" Jul 14 18:04:50.361766 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:50.361673 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b67c8ffe-9e33-4df2-8b8c-a7200eca9764-tls-certs\") pod \"test-model-2d5136-kserve-5468fdc659-d7dh8\" (UID: \"b67c8ffe-9e33-4df2-8b8c-a7200eca9764\") " pod="ai-tenant-e2e-inf-b-2d5136/test-model-2d5136-kserve-5468fdc659-d7dh8" Jul 14 18:04:50.361766 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:50.361701 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b67c8ffe-9e33-4df2-8b8c-a7200eca9764-dshm\") pod \"test-model-2d5136-kserve-5468fdc659-d7dh8\" (UID: \"b67c8ffe-9e33-4df2-8b8c-a7200eca9764\") " pod="ai-tenant-e2e-inf-b-2d5136/test-model-2d5136-kserve-5468fdc659-d7dh8" Jul 14 18:04:50.361766 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:50.361765 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b67c8ffe-9e33-4df2-8b8c-a7200eca9764-tmp-dir\") pod \"test-model-2d5136-kserve-5468fdc659-d7dh8\" (UID: \"b67c8ffe-9e33-4df2-8b8c-a7200eca9764\") " pod="ai-tenant-e2e-inf-b-2d5136/test-model-2d5136-kserve-5468fdc659-d7dh8" Jul 14 18:04:50.362055 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:50.362037 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b67c8ffe-9e33-4df2-8b8c-a7200eca9764-model-cache\") pod \"test-model-2d5136-kserve-5468fdc659-d7dh8\" (UID: \"b67c8ffe-9e33-4df2-8b8c-a7200eca9764\") " pod="ai-tenant-e2e-inf-b-2d5136/test-model-2d5136-kserve-5468fdc659-d7dh8" Jul 14 18:04:50.362127 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:50.362068 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b67c8ffe-9e33-4df2-8b8c-a7200eca9764-home\") pod \"test-model-2d5136-kserve-5468fdc659-d7dh8\" (UID: \"b67c8ffe-9e33-4df2-8b8c-a7200eca9764\") " pod="ai-tenant-e2e-inf-b-2d5136/test-model-2d5136-kserve-5468fdc659-d7dh8" Jul 14 18:04:50.362127 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:50.362114 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b67c8ffe-9e33-4df2-8b8c-a7200eca9764-tmp-dir\") pod \"test-model-2d5136-kserve-5468fdc659-d7dh8\" (UID: \"b67c8ffe-9e33-4df2-8b8c-a7200eca9764\") " pod="ai-tenant-e2e-inf-b-2d5136/test-model-2d5136-kserve-5468fdc659-d7dh8" Jul 14 18:04:50.364033 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:50.364010 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b67c8ffe-9e33-4df2-8b8c-a7200eca9764-dshm\") pod \"test-model-2d5136-kserve-5468fdc659-d7dh8\" (UID: \"b67c8ffe-9e33-4df2-8b8c-a7200eca9764\") " pod="ai-tenant-e2e-inf-b-2d5136/test-model-2d5136-kserve-5468fdc659-d7dh8" Jul 14 18:04:50.364153 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:50.364138 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b67c8ffe-9e33-4df2-8b8c-a7200eca9764-tls-certs\") pod \"test-model-2d5136-kserve-5468fdc659-d7dh8\" (UID: \"b67c8ffe-9e33-4df2-8b8c-a7200eca9764\") " pod="ai-tenant-e2e-inf-b-2d5136/test-model-2d5136-kserve-5468fdc659-d7dh8" Jul 14 18:04:50.370050 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:50.370024 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrhz7\" (UniqueName: \"kubernetes.io/projected/b67c8ffe-9e33-4df2-8b8c-a7200eca9764-kube-api-access-vrhz7\") pod \"test-model-2d5136-kserve-5468fdc659-d7dh8\" (UID: \"b67c8ffe-9e33-4df2-8b8c-a7200eca9764\") " pod="ai-tenant-e2e-inf-b-2d5136/test-model-2d5136-kserve-5468fdc659-d7dh8" Jul 14 18:04:50.507895 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:50.507791 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2d5136/test-model-2d5136-kserve-5468fdc659-d7dh8" Jul 14 18:04:50.642961 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:50.642932 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-2d5136/test-model-2d5136-kserve-5468fdc659-d7dh8"] Jul 14 18:04:50.644882 ip-10-0-132-13 kubenswrapper[2554]: W0714 18:04:50.644854 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb67c8ffe_9e33_4df2_8b8c_a7200eca9764.slice/crio-01ef7af4ec8e640779a907d1a10ea59b50d4d330439be56d71c868d9c8f6da79 WatchSource:0}: Error finding container 01ef7af4ec8e640779a907d1a10ea59b50d4d330439be56d71c868d9c8f6da79: Status 404 returned error can't find the container with id 01ef7af4ec8e640779a907d1a10ea59b50d4d330439be56d71c868d9c8f6da79 Jul 14 18:04:51.282483 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:51.282449 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2d5136/test-model-2d5136-kserve-5468fdc659-d7dh8" event={"ID":"b67c8ffe-9e33-4df2-8b8c-a7200eca9764","Type":"ContainerStarted","Data":"6c0bf91077ad901d1182f0bdcc1045a0b43528a51e376b856b34dc833e08d545"} Jul 14 18:04:51.282869 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:51.282491 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2d5136/test-model-2d5136-kserve-5468fdc659-d7dh8" event={"ID":"b67c8ffe-9e33-4df2-8b8c-a7200eca9764","Type":"ContainerStarted","Data":"01ef7af4ec8e640779a907d1a10ea59b50d4d330439be56d71c868d9c8f6da79"} Jul 14 18:04:51.309131 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:51.305290 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-2d5136/test-model-2d5136-kserve-5468fdc659-d7dh8" podStartSLOduration=1.305250217 podStartE2EDuration="1.305250217s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:04:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:04:51.299803565 +0000 UTC m=+2335.441052291" watchObservedRunningTime="2026-07-14 18:04:51.305250217 +0000 UTC m=+2335.446498922" Jul 14 18:04:55.086763 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:04:55.086726 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:05:00.508234 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:00.508178 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-2d5136/test-model-2d5136-kserve-5468fdc659-d7dh8" Jul 14 18:05:00.508774 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:00.508244 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-2d5136/test-model-2d5136-kserve-5468fdc659-d7dh8" Jul 14 18:05:00.520943 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:00.520921 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-2d5136/test-model-2d5136-kserve-5468fdc659-d7dh8" Jul 14 18:05:01.349008 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:01.348979 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-2d5136/test-model-2d5136-kserve-5468fdc659-d7dh8" Jul 14 18:05:09.225946 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:09.225912 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-2d5136/test-model-2d5136-kserve-5468fdc659-d7dh8"] Jul 14 18:05:09.386050 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:09.386015 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4ef174-szwjv"] Jul 14 18:05:09.391159 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:09.391131 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4ef174-szwjv" Jul 14 18:05:09.394227 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:09.394192 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9rnpk\"" Jul 14 18:05:09.394495 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:09.394454 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4ef174-szwjv"] Jul 14 18:05:09.533270 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:09.533162 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/84789fb0-ff2f-4541-a053-facce71b5200-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4ef174-szwjv\" (UID: \"84789fb0-ff2f-4541-a053-facce71b5200\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4ef174-szwjv" Jul 14 18:05:09.634661 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:09.634630 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/84789fb0-ff2f-4541-a053-facce71b5200-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4ef174-szwjv\" (UID: \"84789fb0-ff2f-4541-a053-facce71b5200\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4ef174-szwjv" Jul 14 18:05:09.635255 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:09.635237 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/84789fb0-ff2f-4541-a053-facce71b5200-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4ef174-szwjv\" (UID: \"84789fb0-ff2f-4541-a053-facce71b5200\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4ef174-szwjv" Jul 14 18:05:09.703353 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:09.703312 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4ef174-szwjv" Jul 14 18:05:09.843177 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:09.843153 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4ef174-szwjv"] Jul 14 18:05:09.844653 ip-10-0-132-13 kubenswrapper[2554]: W0714 18:05:09.844626 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84789fb0_ff2f_4541_a053_facce71b5200.slice/crio-62c91e8542e74936d442b8c108e451613eecc228afaf606fb815f4a75f90eda0 WatchSource:0}: Error finding container 62c91e8542e74936d442b8c108e451613eecc228afaf606fb815f4a75f90eda0: Status 404 returned error can't find the container with id 62c91e8542e74936d442b8c108e451613eecc228afaf606fb815f4a75f90eda0 Jul 14 18:05:10.390552 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:10.390512 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4ef174-szwjv" event={"ID":"84789fb0-ff2f-4541-a053-facce71b5200","Type":"ContainerStarted","Data":"ff9913331b685c2975dd4e6bafc80e09fac6e0b6fd874c662bc0a5e5ca27021d"} Jul 14 18:05:10.390903 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:10.390573 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4ef174-szwjv" event={"ID":"84789fb0-ff2f-4541-a053-facce71b5200","Type":"ContainerStarted","Data":"62c91e8542e74936d442b8c108e451613eecc228afaf606fb815f4a75f90eda0"} Jul 14 18:05:10.410219 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:10.410172 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4ef174-szwjv" podStartSLOduration=1.410155821 podStartE2EDuration="1.410155821s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:05:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:05:10.407721657 +0000 UTC m=+2354.548970360" watchObservedRunningTime="2026-07-14 18:05:10.410155821 +0000 UTC m=+2354.551404525" Jul 14 18:05:11.397491 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:11.397459 2554 generic.go:356] "Generic (PLEG): container finished" podID="84789fb0-ff2f-4541-a053-facce71b5200" containerID="ff9913331b685c2975dd4e6bafc80e09fac6e0b6fd874c662bc0a5e5ca27021d" exitCode=0 Jul 14 18:05:11.397877 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:11.397546 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4ef174-szwjv" event={"ID":"84789fb0-ff2f-4541-a053-facce71b5200","Type":"ContainerDied","Data":"ff9913331b685c2975dd4e6bafc80e09fac6e0b6fd874c662bc0a5e5ca27021d"} Jul 14 18:05:12.528731 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:12.528708 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4ef174-szwjv" Jul 14 18:05:12.662692 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:12.662606 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/84789fb0-ff2f-4541-a053-facce71b5200-maas-api-service-ca\") pod \"84789fb0-ff2f-4541-a053-facce71b5200\" (UID: \"84789fb0-ff2f-4541-a053-facce71b5200\") " Jul 14 18:05:12.662963 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:12.662942 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84789fb0-ff2f-4541-a053-facce71b5200-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "84789fb0-ff2f-4541-a053-facce71b5200" (UID: "84789fb0-ff2f-4541-a053-facce71b5200"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 18:05:12.764108 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:12.764079 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/84789fb0-ff2f-4541-a053-facce71b5200-maas-api-service-ca\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:05:13.409346 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:13.409316 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4ef174-szwjv" Jul 14 18:05:13.409346 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:13.409334 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4ef174-szwjv" event={"ID":"84789fb0-ff2f-4541-a053-facce71b5200","Type":"ContainerDied","Data":"62c91e8542e74936d442b8c108e451613eecc228afaf606fb815f4a75f90eda0"} Jul 14 18:05:13.409552 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:13.409378 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="62c91e8542e74936d442b8c108e451613eecc228afaf606fb815f4a75f90eda0" Jul 14 18:05:13.743076 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:13.742997 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:05:19.870370 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:19.870340 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc"] Jul 14 18:05:19.870767 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:19.870540 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc" podUID="e6515195-58f0-4e48-b73a-a71df779dd1e" containerName="maas-api" containerID="cri-o://89dea6fa38b541f405c4bc48e7cad576c11939e722ca2ce3c8ed2af5173f079f" gracePeriod=30 Jul 14 18:05:20.118207 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:20.118183 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc" Jul 14 18:05:20.231144 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:20.231040 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6515195-58f0-4e48-b73a-a71df779dd1e-openshift-service-ca\") pod \"e6515195-58f0-4e48-b73a-a71df779dd1e\" (UID: \"e6515195-58f0-4e48-b73a-a71df779dd1e\") " Jul 14 18:05:20.231144 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:20.231115 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6515195-58f0-4e48-b73a-a71df779dd1e-maas-api-tls\") pod \"e6515195-58f0-4e48-b73a-a71df779dd1e\" (UID: \"e6515195-58f0-4e48-b73a-a71df779dd1e\") " Jul 14 18:05:20.231428 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:20.231163 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mw5nv\" (UniqueName: \"kubernetes.io/projected/e6515195-58f0-4e48-b73a-a71df779dd1e-kube-api-access-mw5nv\") pod \"e6515195-58f0-4e48-b73a-a71df779dd1e\" (UID: \"e6515195-58f0-4e48-b73a-a71df779dd1e\") " Jul 14 18:05:20.231485 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:20.231454 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6515195-58f0-4e48-b73a-a71df779dd1e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e6515195-58f0-4e48-b73a-a71df779dd1e" (UID: "e6515195-58f0-4e48-b73a-a71df779dd1e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 18:05:20.233238 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:20.233197 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6515195-58f0-4e48-b73a-a71df779dd1e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e6515195-58f0-4e48-b73a-a71df779dd1e" (UID: "e6515195-58f0-4e48-b73a-a71df779dd1e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 18:05:20.233312 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:20.233265 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6515195-58f0-4e48-b73a-a71df779dd1e-kube-api-access-mw5nv" (OuterVolumeSpecName: "kube-api-access-mw5nv") pod "e6515195-58f0-4e48-b73a-a71df779dd1e" (UID: "e6515195-58f0-4e48-b73a-a71df779dd1e"). InnerVolumeSpecName "kube-api-access-mw5nv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:05:20.332467 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:20.332422 2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6515195-58f0-4e48-b73a-a71df779dd1e-openshift-service-ca\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:05:20.332467 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:20.332454 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6515195-58f0-4e48-b73a-a71df779dd1e-maas-api-tls\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:05:20.332467 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:20.332475 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mw5nv\" (UniqueName: \"kubernetes.io/projected/e6515195-58f0-4e48-b73a-a71df779dd1e-kube-api-access-mw5nv\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:05:20.447701 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:20.447668 2554 generic.go:356] "Generic (PLEG): container finished" podID="e6515195-58f0-4e48-b73a-a71df779dd1e" containerID="89dea6fa38b541f405c4bc48e7cad576c11939e722ca2ce3c8ed2af5173f079f" exitCode=0 Jul 14 18:05:20.447861 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:20.447724 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc" Jul 14 18:05:20.447861 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:20.447757 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc" event={"ID":"e6515195-58f0-4e48-b73a-a71df779dd1e","Type":"ContainerDied","Data":"89dea6fa38b541f405c4bc48e7cad576c11939e722ca2ce3c8ed2af5173f079f"} Jul 14 18:05:20.447861 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:20.447812 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc" event={"ID":"e6515195-58f0-4e48-b73a-a71df779dd1e","Type":"ContainerDied","Data":"f265fbf1ff85f2ba5b95fd69929c604cd68ccbac9e274f2d0aa821cc78e66730"} Jul 14 18:05:20.447861 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:20.447840 2554 scope.go:117] "RemoveContainer" containerID="89dea6fa38b541f405c4bc48e7cad576c11939e722ca2ce3c8ed2af5173f079f" Jul 14 18:05:20.456625 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:20.456607 2554 scope.go:117] "RemoveContainer" containerID="89dea6fa38b541f405c4bc48e7cad576c11939e722ca2ce3c8ed2af5173f079f" Jul 14 18:05:20.456888 ip-10-0-132-13 kubenswrapper[2554]: E0714 18:05:20.456865 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89dea6fa38b541f405c4bc48e7cad576c11939e722ca2ce3c8ed2af5173f079f\": container with ID starting with 89dea6fa38b541f405c4bc48e7cad576c11939e722ca2ce3c8ed2af5173f079f not found: ID does not exist" containerID="89dea6fa38b541f405c4bc48e7cad576c11939e722ca2ce3c8ed2af5173f079f" Jul 14 18:05:20.456931 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:20.456896 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89dea6fa38b541f405c4bc48e7cad576c11939e722ca2ce3c8ed2af5173f079f"} err="failed to get container status \"89dea6fa38b541f405c4bc48e7cad576c11939e722ca2ce3c8ed2af5173f079f\": rpc error: code = NotFound desc = could not find container \"89dea6fa38b541f405c4bc48e7cad576c11939e722ca2ce3c8ed2af5173f079f\": container with ID starting with 89dea6fa38b541f405c4bc48e7cad576c11939e722ca2ce3c8ed2af5173f079f not found: ID does not exist" Jul 14 18:05:20.467625 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:20.467599 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc"] Jul 14 18:05:20.471695 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:20.471672 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc"] Jul 14 18:05:22.388260 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:22.388204 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6515195-58f0-4e48-b73a-a71df779dd1e" path="/var/lib/kubelet/pods/e6515195-58f0-4e48-b73a-a71df779dd1e/volumes" Jul 14 18:05:23.060610 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:23.060556 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4ef174-f59cf7b5b-zgxrc" podUID="e6515195-58f0-4e48-b73a-a71df779dd1e" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.54:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Jul 14 18:05:24.245791 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:24.245747 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-2d5136/test-model-2d5136-kserve-5468fdc659-d7dh8" podUID="b67c8ffe-9e33-4df2-8b8c-a7200eca9764" containerName="main" containerID="cri-o://6c0bf91077ad901d1182f0bdcc1045a0b43528a51e376b856b34dc833e08d545" gracePeriod=45 Jul 14 18:05:24.471131 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:24.471102 2554 generic.go:356] "Generic (PLEG): container finished" podID="b67c8ffe-9e33-4df2-8b8c-a7200eca9764" containerID="6c0bf91077ad901d1182f0bdcc1045a0b43528a51e376b856b34dc833e08d545" exitCode=0 Jul 14 18:05:24.471310 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:24.471170 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2d5136/test-model-2d5136-kserve-5468fdc659-d7dh8" event={"ID":"b67c8ffe-9e33-4df2-8b8c-a7200eca9764","Type":"ContainerDied","Data":"6c0bf91077ad901d1182f0bdcc1045a0b43528a51e376b856b34dc833e08d545"} Jul 14 18:05:24.509723 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:24.509665 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2d5136/test-model-2d5136-kserve-5468fdc659-d7dh8" Jul 14 18:05:24.569601 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:24.569565 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b67c8ffe-9e33-4df2-8b8c-a7200eca9764-tls-certs\") pod \"b67c8ffe-9e33-4df2-8b8c-a7200eca9764\" (UID: \"b67c8ffe-9e33-4df2-8b8c-a7200eca9764\") " Jul 14 18:05:24.569777 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:24.569612 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrhz7\" (UniqueName: \"kubernetes.io/projected/b67c8ffe-9e33-4df2-8b8c-a7200eca9764-kube-api-access-vrhz7\") pod \"b67c8ffe-9e33-4df2-8b8c-a7200eca9764\" (UID: \"b67c8ffe-9e33-4df2-8b8c-a7200eca9764\") " Jul 14 18:05:24.569777 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:24.569656 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b67c8ffe-9e33-4df2-8b8c-a7200eca9764-tmp-dir\") pod \"b67c8ffe-9e33-4df2-8b8c-a7200eca9764\" (UID: \"b67c8ffe-9e33-4df2-8b8c-a7200eca9764\") " Jul 14 18:05:24.569777 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:24.569675 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b67c8ffe-9e33-4df2-8b8c-a7200eca9764-model-cache\") pod \"b67c8ffe-9e33-4df2-8b8c-a7200eca9764\" (UID: \"b67c8ffe-9e33-4df2-8b8c-a7200eca9764\") " Jul 14 18:05:24.569777 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:24.569721 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b67c8ffe-9e33-4df2-8b8c-a7200eca9764-dshm\") pod \"b67c8ffe-9e33-4df2-8b8c-a7200eca9764\" (UID: \"b67c8ffe-9e33-4df2-8b8c-a7200eca9764\") " Jul 14 18:05:24.569777 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:24.569749 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b67c8ffe-9e33-4df2-8b8c-a7200eca9764-home\") pod \"b67c8ffe-9e33-4df2-8b8c-a7200eca9764\" (UID: \"b67c8ffe-9e33-4df2-8b8c-a7200eca9764\") " Jul 14 18:05:24.570052 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:24.569966 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b67c8ffe-9e33-4df2-8b8c-a7200eca9764-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b67c8ffe-9e33-4df2-8b8c-a7200eca9764" (UID: "b67c8ffe-9e33-4df2-8b8c-a7200eca9764"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:05:24.570052 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:24.569995 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b67c8ffe-9e33-4df2-8b8c-a7200eca9764-model-cache" (OuterVolumeSpecName: "model-cache") pod "b67c8ffe-9e33-4df2-8b8c-a7200eca9764" (UID: "b67c8ffe-9e33-4df2-8b8c-a7200eca9764"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:05:24.570130 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:24.570049 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b67c8ffe-9e33-4df2-8b8c-a7200eca9764-home" (OuterVolumeSpecName: "home") pod "b67c8ffe-9e33-4df2-8b8c-a7200eca9764" (UID: "b67c8ffe-9e33-4df2-8b8c-a7200eca9764"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:05:24.571940 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:24.571904 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b67c8ffe-9e33-4df2-8b8c-a7200eca9764-dshm" (OuterVolumeSpecName: "dshm") pod "b67c8ffe-9e33-4df2-8b8c-a7200eca9764" (UID: "b67c8ffe-9e33-4df2-8b8c-a7200eca9764"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:05:24.571940 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:24.571919 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b67c8ffe-9e33-4df2-8b8c-a7200eca9764-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b67c8ffe-9e33-4df2-8b8c-a7200eca9764" (UID: "b67c8ffe-9e33-4df2-8b8c-a7200eca9764"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 18:05:24.572087 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:24.571959 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b67c8ffe-9e33-4df2-8b8c-a7200eca9764-kube-api-access-vrhz7" (OuterVolumeSpecName: "kube-api-access-vrhz7") pod "b67c8ffe-9e33-4df2-8b8c-a7200eca9764" (UID: "b67c8ffe-9e33-4df2-8b8c-a7200eca9764"). InnerVolumeSpecName "kube-api-access-vrhz7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:05:24.670936 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:24.670898 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrhz7\" (UniqueName: \"kubernetes.io/projected/b67c8ffe-9e33-4df2-8b8c-a7200eca9764-kube-api-access-vrhz7\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:05:24.670936 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:24.670930 2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b67c8ffe-9e33-4df2-8b8c-a7200eca9764-tmp-dir\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:05:24.670936 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:24.670943 2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b67c8ffe-9e33-4df2-8b8c-a7200eca9764-model-cache\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:05:24.671174 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:24.670955 2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b67c8ffe-9e33-4df2-8b8c-a7200eca9764-dshm\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:05:24.671174 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:24.670966 2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b67c8ffe-9e33-4df2-8b8c-a7200eca9764-home\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:05:24.671174 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:24.670975 2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b67c8ffe-9e33-4df2-8b8c-a7200eca9764-tls-certs\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:05:24.751273 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:24.751243 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:05:25.477635 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:25.477603 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2d5136/test-model-2d5136-kserve-5468fdc659-d7dh8" Jul 14 18:05:25.477635 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:25.477618 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2d5136/test-model-2d5136-kserve-5468fdc659-d7dh8" event={"ID":"b67c8ffe-9e33-4df2-8b8c-a7200eca9764","Type":"ContainerDied","Data":"01ef7af4ec8e640779a907d1a10ea59b50d4d330439be56d71c868d9c8f6da79"} Jul 14 18:05:25.478138 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:25.477665 2554 scope.go:117] "RemoveContainer" containerID="6c0bf91077ad901d1182f0bdcc1045a0b43528a51e376b856b34dc833e08d545" Jul 14 18:05:25.509653 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:25.509621 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-2d5136/test-model-2d5136-kserve-5468fdc659-d7dh8"] Jul 14 18:05:25.521892 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:25.521868 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-2d5136/test-model-2d5136-kserve-5468fdc659-d7dh8"] Jul 14 18:05:26.388139 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:26.388105 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b67c8ffe-9e33-4df2-8b8c-a7200eca9764" path="/var/lib/kubelet/pods/b67c8ffe-9e33-4df2-8b8c-a7200eca9764/volumes" Jul 14 18:05:35.502705 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:35.502671 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2d5136-qrrkv"] Jul 14 18:05:35.503079 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:35.503002 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84789fb0-ff2f-4541-a053-facce71b5200" containerName="revoke-keys" Jul 14 18:05:35.503079 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:35.503014 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="84789fb0-ff2f-4541-a053-facce71b5200" containerName="revoke-keys" Jul 14 18:05:35.503079 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:35.503026 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6515195-58f0-4e48-b73a-a71df779dd1e" containerName="maas-api" Jul 14 18:05:35.503079 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:35.503032 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6515195-58f0-4e48-b73a-a71df779dd1e" containerName="maas-api" Jul 14 18:05:35.503079 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:35.503042 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b67c8ffe-9e33-4df2-8b8c-a7200eca9764" containerName="main" Jul 14 18:05:35.503079 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:35.503047 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="b67c8ffe-9e33-4df2-8b8c-a7200eca9764" containerName="main" Jul 14 18:05:35.503333 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:35.503110 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6515195-58f0-4e48-b73a-a71df779dd1e" containerName="maas-api" Jul 14 18:05:35.503333 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:35.503118 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="b67c8ffe-9e33-4df2-8b8c-a7200eca9764" containerName="main" Jul 14 18:05:35.503333 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:35.503124 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="84789fb0-ff2f-4541-a053-facce71b5200" containerName="revoke-keys" Jul 14 18:05:35.505965 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:35.505950 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2d5136-qrrkv" Jul 14 18:05:35.509523 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:35.509494 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9rnpk\"" Jul 14 18:05:35.514438 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:35.514415 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2d5136-qrrkv"] Jul 14 18:05:35.569614 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:35.569585 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0b7e330-6661-4e9a-9b97-c0a2079c33f9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-2d5136-qrrkv\" (UID: \"f0b7e330-6661-4e9a-9b97-c0a2079c33f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2d5136-qrrkv" Jul 14 18:05:35.670653 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:35.670621 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0b7e330-6661-4e9a-9b97-c0a2079c33f9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-2d5136-qrrkv\" (UID: \"f0b7e330-6661-4e9a-9b97-c0a2079c33f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2d5136-qrrkv" Jul 14 18:05:35.671191 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:35.671174 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0b7e330-6661-4e9a-9b97-c0a2079c33f9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-2d5136-qrrkv\" (UID: \"f0b7e330-6661-4e9a-9b97-c0a2079c33f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2d5136-qrrkv" Jul 14 18:05:35.817253 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:35.817160 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2d5136-qrrkv" Jul 14 18:05:36.154374 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:36.154341 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2d5136-qrrkv"] Jul 14 18:05:36.154697 ip-10-0-132-13 kubenswrapper[2554]: W0714 18:05:36.154675 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0b7e330_6661_4e9a_9b97_c0a2079c33f9.slice/crio-3fab44250abf0978db28abfcaa38b8e62c3c868a3686dfe41ce9621592563c91 WatchSource:0}: Error finding container 3fab44250abf0978db28abfcaa38b8e62c3c868a3686dfe41ce9621592563c91: Status 404 returned error can't find the container with id 3fab44250abf0978db28abfcaa38b8e62c3c868a3686dfe41ce9621592563c91 Jul 14 18:05:36.539268 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:36.539236 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2d5136-qrrkv" event={"ID":"f0b7e330-6661-4e9a-9b97-c0a2079c33f9","Type":"ContainerStarted","Data":"d4bb53a8e90cb7858c4136cde7978a185961fd3aa5ee093b54b5ca9bc0824f57"} Jul 14 18:05:36.539601 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:36.539292 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2d5136-qrrkv" event={"ID":"f0b7e330-6661-4e9a-9b97-c0a2079c33f9","Type":"ContainerStarted","Data":"3fab44250abf0978db28abfcaa38b8e62c3c868a3686dfe41ce9621592563c91"} Jul 14 18:05:36.558278 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:36.558194 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2d5136-qrrkv" podStartSLOduration=1.558179998 podStartE2EDuration="1.558179998s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:05:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:05:36.554908532 +0000 UTC m=+2380.696157237" watchObservedRunningTime="2026-07-14 18:05:36.558179998 +0000 UTC m=+2380.699428702" Jul 14 18:05:37.547201 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:37.547162 2554 generic.go:356] "Generic (PLEG): container finished" podID="f0b7e330-6661-4e9a-9b97-c0a2079c33f9" containerID="d4bb53a8e90cb7858c4136cde7978a185961fd3aa5ee093b54b5ca9bc0824f57" exitCode=0 Jul 14 18:05:37.547565 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:37.547244 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2d5136-qrrkv" event={"ID":"f0b7e330-6661-4e9a-9b97-c0a2079c33f9","Type":"ContainerDied","Data":"d4bb53a8e90cb7858c4136cde7978a185961fd3aa5ee093b54b5ca9bc0824f57"} Jul 14 18:05:38.681014 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:38.680991 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2d5136-qrrkv" Jul 14 18:05:38.798281 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:38.798251 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0b7e330-6661-4e9a-9b97-c0a2079c33f9-maas-api-service-ca\") pod \"f0b7e330-6661-4e9a-9b97-c0a2079c33f9\" (UID: \"f0b7e330-6661-4e9a-9b97-c0a2079c33f9\") " Jul 14 18:05:38.798606 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:38.798582 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0b7e330-6661-4e9a-9b97-c0a2079c33f9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f0b7e330-6661-4e9a-9b97-c0a2079c33f9" (UID: "f0b7e330-6661-4e9a-9b97-c0a2079c33f9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 18:05:38.899109 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:38.899079 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0b7e330-6661-4e9a-9b97-c0a2079c33f9-maas-api-service-ca\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:05:39.559986 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:39.559960 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2d5136-qrrkv" Jul 14 18:05:39.560159 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:39.560002 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2d5136-qrrkv" event={"ID":"f0b7e330-6661-4e9a-9b97-c0a2079c33f9","Type":"ContainerDied","Data":"3fab44250abf0978db28abfcaa38b8e62c3c868a3686dfe41ce9621592563c91"} Jul 14 18:05:39.560159 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:39.560051 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3fab44250abf0978db28abfcaa38b8e62c3c868a3686dfe41ce9621592563c91" Jul 14 18:05:45.953697 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:45.953659 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-2d5136-76bf549667-6fxjc"] Jul 14 18:05:45.954285 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:45.953938 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2d5136-76bf549667-6fxjc" podUID="1920c592-bac3-4a0f-9665-9689445c36a9" containerName="maas-api" containerID="cri-o://bf4d192f5e2e40d51a9cc6e88e10e0c1b52992f7026b437f59f76231a3a66440" gracePeriod=30 Jul 14 18:05:46.199760 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:46.199738 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2d5136-76bf549667-6fxjc" Jul 14 18:05:46.259740 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:46.259665 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vgdr5\" (UniqueName: \"kubernetes.io/projected/1920c592-bac3-4a0f-9665-9689445c36a9-kube-api-access-vgdr5\") pod \"1920c592-bac3-4a0f-9665-9689445c36a9\" (UID: \"1920c592-bac3-4a0f-9665-9689445c36a9\") " Jul 14 18:05:46.259740 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:46.259706 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1920c592-bac3-4a0f-9665-9689445c36a9-openshift-service-ca\") pod \"1920c592-bac3-4a0f-9665-9689445c36a9\" (UID: \"1920c592-bac3-4a0f-9665-9689445c36a9\") " Jul 14 18:05:46.259960 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:46.259783 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1920c592-bac3-4a0f-9665-9689445c36a9-maas-api-tls\") pod \"1920c592-bac3-4a0f-9665-9689445c36a9\" (UID: \"1920c592-bac3-4a0f-9665-9689445c36a9\") " Jul 14 18:05:46.260078 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:46.260056 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1920c592-bac3-4a0f-9665-9689445c36a9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1920c592-bac3-4a0f-9665-9689445c36a9" (UID: "1920c592-bac3-4a0f-9665-9689445c36a9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 18:05:46.261692 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:46.261668 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1920c592-bac3-4a0f-9665-9689445c36a9-kube-api-access-vgdr5" (OuterVolumeSpecName: "kube-api-access-vgdr5") pod "1920c592-bac3-4a0f-9665-9689445c36a9" (UID: "1920c592-bac3-4a0f-9665-9689445c36a9"). InnerVolumeSpecName "kube-api-access-vgdr5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:05:46.261847 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:46.261827 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1920c592-bac3-4a0f-9665-9689445c36a9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1920c592-bac3-4a0f-9665-9689445c36a9" (UID: "1920c592-bac3-4a0f-9665-9689445c36a9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 18:05:46.361270 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:46.361241 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1920c592-bac3-4a0f-9665-9689445c36a9-maas-api-tls\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:05:46.361270 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:46.361267 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vgdr5\" (UniqueName: \"kubernetes.io/projected/1920c592-bac3-4a0f-9665-9689445c36a9-kube-api-access-vgdr5\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:05:46.361435 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:46.361277 2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1920c592-bac3-4a0f-9665-9689445c36a9-openshift-service-ca\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:05:46.597676 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:46.597593 2554 generic.go:356] "Generic (PLEG): container finished" podID="1920c592-bac3-4a0f-9665-9689445c36a9" containerID="bf4d192f5e2e40d51a9cc6e88e10e0c1b52992f7026b437f59f76231a3a66440" exitCode=0 Jul 14 18:05:46.597676 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:46.597643 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2d5136-76bf549667-6fxjc" Jul 14 18:05:46.597861 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:46.597681 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2d5136-76bf549667-6fxjc" event={"ID":"1920c592-bac3-4a0f-9665-9689445c36a9","Type":"ContainerDied","Data":"bf4d192f5e2e40d51a9cc6e88e10e0c1b52992f7026b437f59f76231a3a66440"} Jul 14 18:05:46.597861 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:46.597738 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2d5136-76bf549667-6fxjc" event={"ID":"1920c592-bac3-4a0f-9665-9689445c36a9","Type":"ContainerDied","Data":"7363dfc5e94009867cc9d34df8de5a8fb42a0492381215ebe629fa2258283560"} Jul 14 18:05:46.597861 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:46.597768 2554 scope.go:117] "RemoveContainer" containerID="bf4d192f5e2e40d51a9cc6e88e10e0c1b52992f7026b437f59f76231a3a66440" Jul 14 18:05:46.606671 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:46.606659 2554 scope.go:117] "RemoveContainer" containerID="bf4d192f5e2e40d51a9cc6e88e10e0c1b52992f7026b437f59f76231a3a66440" Jul 14 18:05:46.606952 ip-10-0-132-13 kubenswrapper[2554]: E0714 18:05:46.606934 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf4d192f5e2e40d51a9cc6e88e10e0c1b52992f7026b437f59f76231a3a66440\": container with ID starting with bf4d192f5e2e40d51a9cc6e88e10e0c1b52992f7026b437f59f76231a3a66440 not found: ID does not exist" containerID="bf4d192f5e2e40d51a9cc6e88e10e0c1b52992f7026b437f59f76231a3a66440" Jul 14 18:05:46.607002 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:46.606962 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf4d192f5e2e40d51a9cc6e88e10e0c1b52992f7026b437f59f76231a3a66440"} err="failed to get container status \"bf4d192f5e2e40d51a9cc6e88e10e0c1b52992f7026b437f59f76231a3a66440\": rpc error: code = NotFound desc = could not find container \"bf4d192f5e2e40d51a9cc6e88e10e0c1b52992f7026b437f59f76231a3a66440\": container with ID starting with bf4d192f5e2e40d51a9cc6e88e10e0c1b52992f7026b437f59f76231a3a66440 not found: ID does not exist" Jul 14 18:05:46.615906 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:46.615882 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-2d5136-76bf549667-6fxjc"] Jul 14 18:05:46.619786 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:46.619766 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-2d5136-76bf549667-6fxjc"] Jul 14 18:05:48.387529 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:48.387494 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1920c592-bac3-4a0f-9665-9689445c36a9" path="/var/lib/kubelet/pods/1920c592-bac3-4a0f-9665-9689445c36a9/volumes" Jul 14 18:05:49.258666 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:49.258631 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:05:56.470403 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:56.470375 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pk9h2_f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726/ovn-acl-logging/0.log" Jul 14 18:05:56.477884 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:05:56.477863 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pk9h2_f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726/ovn-acl-logging/0.log" Jul 14 18:06:05.868002 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:05.867969 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e8afdc-894846795-6wfk2"] Jul 14 18:06:05.868424 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:05.868339 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1920c592-bac3-4a0f-9665-9689445c36a9" containerName="maas-api" Jul 14 18:06:05.868424 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:05.868350 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="1920c592-bac3-4a0f-9665-9689445c36a9" containerName="maas-api" Jul 14 18:06:05.868424 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:05.868358 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0b7e330-6661-4e9a-9b97-c0a2079c33f9" containerName="revoke-keys" Jul 14 18:06:05.868424 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:05.868363 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0b7e330-6661-4e9a-9b97-c0a2079c33f9" containerName="revoke-keys" Jul 14 18:06:05.868424 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:05.868413 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0b7e330-6661-4e9a-9b97-c0a2079c33f9" containerName="revoke-keys" Jul 14 18:06:05.868424 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:05.868423 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="1920c592-bac3-4a0f-9665-9689445c36a9" containerName="maas-api" Jul 14 18:06:05.871380 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:05.871361 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8afdc-894846795-6wfk2" Jul 14 18:06:05.874297 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:05.874276 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 18:06:05.874543 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:05.874523 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 14 18:06:05.874632 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:05.874545 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-e8afdc-serving-cert\"" Jul 14 18:06:05.875359 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:05.875333 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-ggvwm\"" Jul 14 18:06:05.882200 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:05.882179 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e8afdc-894846795-6wfk2"] Jul 14 18:06:06.024920 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:06.024882 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vs47t\" (UniqueName: \"kubernetes.io/projected/738a4ecb-925e-4159-a8fc-02f1961e1f03-kube-api-access-vs47t\") pod \"maas-api-e2e-api-a-e8afdc-894846795-6wfk2\" (UID: \"738a4ecb-925e-4159-a8fc-02f1961e1f03\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8afdc-894846795-6wfk2" Jul 14 18:06:06.025100 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:06.024944 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/738a4ecb-925e-4159-a8fc-02f1961e1f03-openshift-service-ca\") pod \"maas-api-e2e-api-a-e8afdc-894846795-6wfk2\" (UID: \"738a4ecb-925e-4159-a8fc-02f1961e1f03\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8afdc-894846795-6wfk2" Jul 14 18:06:06.025100 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:06.025036 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/738a4ecb-925e-4159-a8fc-02f1961e1f03-maas-api-tls\") pod \"maas-api-e2e-api-a-e8afdc-894846795-6wfk2\" (UID: \"738a4ecb-925e-4159-a8fc-02f1961e1f03\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8afdc-894846795-6wfk2" Jul 14 18:06:06.125928 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:06.125836 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/738a4ecb-925e-4159-a8fc-02f1961e1f03-openshift-service-ca\") pod \"maas-api-e2e-api-a-e8afdc-894846795-6wfk2\" (UID: \"738a4ecb-925e-4159-a8fc-02f1961e1f03\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8afdc-894846795-6wfk2" Jul 14 18:06:06.125928 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:06.125895 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/738a4ecb-925e-4159-a8fc-02f1961e1f03-maas-api-tls\") pod \"maas-api-e2e-api-a-e8afdc-894846795-6wfk2\" (UID: \"738a4ecb-925e-4159-a8fc-02f1961e1f03\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8afdc-894846795-6wfk2" Jul 14 18:06:06.125928 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:06.125929 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vs47t\" (UniqueName: \"kubernetes.io/projected/738a4ecb-925e-4159-a8fc-02f1961e1f03-kube-api-access-vs47t\") pod \"maas-api-e2e-api-a-e8afdc-894846795-6wfk2\" (UID: \"738a4ecb-925e-4159-a8fc-02f1961e1f03\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8afdc-894846795-6wfk2" Jul 14 18:06:06.126546 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:06.126528 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/738a4ecb-925e-4159-a8fc-02f1961e1f03-openshift-service-ca\") pod \"maas-api-e2e-api-a-e8afdc-894846795-6wfk2\" (UID: \"738a4ecb-925e-4159-a8fc-02f1961e1f03\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8afdc-894846795-6wfk2" Jul 14 18:06:06.128448 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:06.128421 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/738a4ecb-925e-4159-a8fc-02f1961e1f03-maas-api-tls\") pod \"maas-api-e2e-api-a-e8afdc-894846795-6wfk2\" (UID: \"738a4ecb-925e-4159-a8fc-02f1961e1f03\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8afdc-894846795-6wfk2" Jul 14 18:06:06.136197 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:06.136176 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vs47t\" (UniqueName: \"kubernetes.io/projected/738a4ecb-925e-4159-a8fc-02f1961e1f03-kube-api-access-vs47t\") pod \"maas-api-e2e-api-a-e8afdc-894846795-6wfk2\" (UID: \"738a4ecb-925e-4159-a8fc-02f1961e1f03\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8afdc-894846795-6wfk2" Jul 14 18:06:06.183357 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:06.183319 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8afdc-894846795-6wfk2" Jul 14 18:06:06.321883 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:06.321854 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e8afdc-894846795-6wfk2"] Jul 14 18:06:06.322718 ip-10-0-132-13 kubenswrapper[2554]: W0714 18:06:06.322695 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod738a4ecb_925e_4159_a8fc_02f1961e1f03.slice/crio-494dfcc4171a127e59caef9d6e071469ae05af68aba42ccdd0ae6d31ec4fdf90 WatchSource:0}: Error finding container 494dfcc4171a127e59caef9d6e071469ae05af68aba42ccdd0ae6d31ec4fdf90: Status 404 returned error can't find the container with id 494dfcc4171a127e59caef9d6e071469ae05af68aba42ccdd0ae6d31ec4fdf90 Jul 14 18:06:06.324120 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:06.324104 2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 18:06:06.710696 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:06.710659 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8afdc-894846795-6wfk2" event={"ID":"738a4ecb-925e-4159-a8fc-02f1961e1f03","Type":"ContainerStarted","Data":"494dfcc4171a127e59caef9d6e071469ae05af68aba42ccdd0ae6d31ec4fdf90"} Jul 14 18:06:07.717958 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:07.717912 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8afdc-894846795-6wfk2" event={"ID":"738a4ecb-925e-4159-a8fc-02f1961e1f03","Type":"ContainerStarted","Data":"3913c0301899231f07f140863b6e817215bf90e0ca46e1d0c736e3c67969f32c"} Jul 14 18:06:07.718398 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:07.718000 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8afdc-894846795-6wfk2" Jul 14 18:06:07.738567 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:07.738520 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8afdc-894846795-6wfk2" podStartSLOduration=1.9668444630000002 podStartE2EDuration="2.738505137s" totalImagesPullingTime="771.660674ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:06:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:06:07.737558514 +0000 UTC m=+2411.878807221" watchObservedRunningTime="2026-07-14 18:06:07.738505137 +0000 UTC m=+2411.879753887" Jul 14 18:06:10.754662 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:10.754626 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:06:13.729359 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:13.729328 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8afdc-894846795-6wfk2" Jul 14 18:06:13.785346 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:13.785311 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e9d9a7-66695689d6-2jclj"] Jul 14 18:06:13.789031 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:13.789007 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e9d9a7-66695689d6-2jclj" Jul 14 18:06:13.793058 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:13.793039 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-e9d9a7-serving-cert\"" Jul 14 18:06:13.798547 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:13.798521 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01b9c566-8d5c-4fc0-9cb3-81f13958784d-maas-api-tls\") pod \"maas-api-e2e-api-b-e9d9a7-66695689d6-2jclj\" (UID: \"01b9c566-8d5c-4fc0-9cb3-81f13958784d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e9d9a7-66695689d6-2jclj" Jul 14 18:06:13.798674 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:13.798654 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bnck\" (UniqueName: \"kubernetes.io/projected/01b9c566-8d5c-4fc0-9cb3-81f13958784d-kube-api-access-7bnck\") pod \"maas-api-e2e-api-b-e9d9a7-66695689d6-2jclj\" (UID: \"01b9c566-8d5c-4fc0-9cb3-81f13958784d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e9d9a7-66695689d6-2jclj" Jul 14 18:06:13.798737 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:13.798702 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01b9c566-8d5c-4fc0-9cb3-81f13958784d-openshift-service-ca\") pod \"maas-api-e2e-api-b-e9d9a7-66695689d6-2jclj\" (UID: \"01b9c566-8d5c-4fc0-9cb3-81f13958784d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e9d9a7-66695689d6-2jclj" Jul 14 18:06:13.804641 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:13.804615 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e9d9a7-66695689d6-2jclj"] Jul 14 18:06:13.899455 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:13.899417 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bnck\" (UniqueName: \"kubernetes.io/projected/01b9c566-8d5c-4fc0-9cb3-81f13958784d-kube-api-access-7bnck\") pod \"maas-api-e2e-api-b-e9d9a7-66695689d6-2jclj\" (UID: \"01b9c566-8d5c-4fc0-9cb3-81f13958784d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e9d9a7-66695689d6-2jclj" Jul 14 18:06:13.899651 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:13.899468 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01b9c566-8d5c-4fc0-9cb3-81f13958784d-openshift-service-ca\") pod \"maas-api-e2e-api-b-e9d9a7-66695689d6-2jclj\" (UID: \"01b9c566-8d5c-4fc0-9cb3-81f13958784d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e9d9a7-66695689d6-2jclj" Jul 14 18:06:13.899651 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:13.899538 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01b9c566-8d5c-4fc0-9cb3-81f13958784d-maas-api-tls\") pod \"maas-api-e2e-api-b-e9d9a7-66695689d6-2jclj\" (UID: \"01b9c566-8d5c-4fc0-9cb3-81f13958784d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e9d9a7-66695689d6-2jclj" Jul 14 18:06:13.900124 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:13.900103 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01b9c566-8d5c-4fc0-9cb3-81f13958784d-openshift-service-ca\") pod \"maas-api-e2e-api-b-e9d9a7-66695689d6-2jclj\" (UID: \"01b9c566-8d5c-4fc0-9cb3-81f13958784d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e9d9a7-66695689d6-2jclj" Jul 14 18:06:13.901939 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:13.901914 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01b9c566-8d5c-4fc0-9cb3-81f13958784d-maas-api-tls\") pod \"maas-api-e2e-api-b-e9d9a7-66695689d6-2jclj\" (UID: \"01b9c566-8d5c-4fc0-9cb3-81f13958784d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e9d9a7-66695689d6-2jclj" Jul 14 18:06:13.909111 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:13.909093 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bnck\" (UniqueName: \"kubernetes.io/projected/01b9c566-8d5c-4fc0-9cb3-81f13958784d-kube-api-access-7bnck\") pod \"maas-api-e2e-api-b-e9d9a7-66695689d6-2jclj\" (UID: \"01b9c566-8d5c-4fc0-9cb3-81f13958784d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e9d9a7-66695689d6-2jclj" Jul 14 18:06:14.099565 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:14.099484 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e9d9a7-66695689d6-2jclj" Jul 14 18:06:14.251942 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:14.251908 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e9d9a7-66695689d6-2jclj"] Jul 14 18:06:14.254691 ip-10-0-132-13 kubenswrapper[2554]: W0714 18:06:14.254665 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01b9c566_8d5c_4fc0_9cb3_81f13958784d.slice/crio-e4cc77be90912446f14815210bcd973517f61fc79754ff9aa91a46426086bc69 WatchSource:0}: Error finding container e4cc77be90912446f14815210bcd973517f61fc79754ff9aa91a46426086bc69: Status 404 returned error can't find the container with id e4cc77be90912446f14815210bcd973517f61fc79754ff9aa91a46426086bc69 Jul 14 18:06:14.764654 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:14.764614 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e9d9a7-66695689d6-2jclj" event={"ID":"01b9c566-8d5c-4fc0-9cb3-81f13958784d","Type":"ContainerStarted","Data":"2ee5ea81ee61a019ee15bdb8fa57021d3e86f2249c9e093a6f9ced8852f04de7"} Jul 14 18:06:14.765039 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:14.764695 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e9d9a7-66695689d6-2jclj" event={"ID":"01b9c566-8d5c-4fc0-9cb3-81f13958784d","Type":"ContainerStarted","Data":"e4cc77be90912446f14815210bcd973517f61fc79754ff9aa91a46426086bc69"} Jul 14 18:06:14.765039 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:14.764736 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e9d9a7-66695689d6-2jclj" Jul 14 18:06:14.787283 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:14.787223 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e9d9a7-66695689d6-2jclj" podStartSLOduration=1.383515954 podStartE2EDuration="1.787194588s" totalImagesPullingTime="403.678634ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:06:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:06:14.784453998 +0000 UTC m=+2418.925702716" watchObservedRunningTime="2026-07-14 18:06:14.787194588 +0000 UTC m=+2418.928443292" Jul 14 18:06:20.775856 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:20.775784 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e9d9a7-66695689d6-2jclj" Jul 14 18:06:26.014829 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:26.014792 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8afdc-gtjvs"] Jul 14 18:06:26.018302 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:26.018287 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8afdc-gtjvs" Jul 14 18:06:26.020939 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:26.020917 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9rnpk\"" Jul 14 18:06:26.026493 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:26.026472 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8afdc-gtjvs"] Jul 14 18:06:26.101727 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:26.101695 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d08e2e3-02bf-4809-9ddf-612135c667b5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-e8afdc-gtjvs\" (UID: \"9d08e2e3-02bf-4809-9ddf-612135c667b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8afdc-gtjvs" Jul 14 18:06:26.202921 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:26.202888 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d08e2e3-02bf-4809-9ddf-612135c667b5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-e8afdc-gtjvs\" (UID: \"9d08e2e3-02bf-4809-9ddf-612135c667b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8afdc-gtjvs" Jul 14 18:06:26.203491 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:26.203474 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d08e2e3-02bf-4809-9ddf-612135c667b5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-e8afdc-gtjvs\" (UID: \"9d08e2e3-02bf-4809-9ddf-612135c667b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8afdc-gtjvs" Jul 14 18:06:26.330840 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:26.330765 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8afdc-gtjvs" Jul 14 18:06:26.466359 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:26.466330 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8afdc-gtjvs"] Jul 14 18:06:26.467803 ip-10-0-132-13 kubenswrapper[2554]: W0714 18:06:26.467772 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d08e2e3_02bf_4809_9ddf_612135c667b5.slice/crio-e6ecbdb42ef29f3412c7581f4add17629e23e25579ad7c394e06b940d8469596 WatchSource:0}: Error finding container e6ecbdb42ef29f3412c7581f4add17629e23e25579ad7c394e06b940d8469596: Status 404 returned error can't find the container with id e6ecbdb42ef29f3412c7581f4add17629e23e25579ad7c394e06b940d8469596 Jul 14 18:06:26.833168 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:26.833128 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8afdc-gtjvs" event={"ID":"9d08e2e3-02bf-4809-9ddf-612135c667b5","Type":"ContainerStarted","Data":"805cdb3efa9897728691f7cfa843f2b659cf75378f42dfac8611dfe8bb2350ae"} Jul 14 18:06:26.833337 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:26.833189 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8afdc-gtjvs" event={"ID":"9d08e2e3-02bf-4809-9ddf-612135c667b5","Type":"ContainerStarted","Data":"e6ecbdb42ef29f3412c7581f4add17629e23e25579ad7c394e06b940d8469596"} Jul 14 18:06:26.853697 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:26.853640 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8afdc-gtjvs" podStartSLOduration=1.853620356 podStartE2EDuration="1.853620356s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:06:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:06:26.853002307 +0000 UTC m=+2430.994251039" watchObservedRunningTime="2026-07-14 18:06:26.853620356 +0000 UTC m=+2430.994869061" Jul 14 18:06:27.045627 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:27.045540 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:06:27.839618 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:27.839583 2554 generic.go:356] "Generic (PLEG): container finished" podID="9d08e2e3-02bf-4809-9ddf-612135c667b5" containerID="805cdb3efa9897728691f7cfa843f2b659cf75378f42dfac8611dfe8bb2350ae" exitCode=0 Jul 14 18:06:27.839819 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:27.839630 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8afdc-gtjvs" event={"ID":"9d08e2e3-02bf-4809-9ddf-612135c667b5","Type":"ContainerDied","Data":"805cdb3efa9897728691f7cfa843f2b659cf75378f42dfac8611dfe8bb2350ae"} Jul 14 18:06:28.977104 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:28.977084 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8afdc-gtjvs" Jul 14 18:06:29.025281 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:29.025250 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d08e2e3-02bf-4809-9ddf-612135c667b5-maas-api-service-ca\") pod \"9d08e2e3-02bf-4809-9ddf-612135c667b5\" (UID: \"9d08e2e3-02bf-4809-9ddf-612135c667b5\") " Jul 14 18:06:29.025556 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:29.025535 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d08e2e3-02bf-4809-9ddf-612135c667b5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9d08e2e3-02bf-4809-9ddf-612135c667b5" (UID: "9d08e2e3-02bf-4809-9ddf-612135c667b5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 18:06:29.125871 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:29.125839 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d08e2e3-02bf-4809-9ddf-612135c667b5-maas-api-service-ca\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:06:29.850518 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:29.850493 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8afdc-gtjvs" Jul 14 18:06:29.850518 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:29.850516 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8afdc-gtjvs" event={"ID":"9d08e2e3-02bf-4809-9ddf-612135c667b5","Type":"ContainerDied","Data":"e6ecbdb42ef29f3412c7581f4add17629e23e25579ad7c394e06b940d8469596"} Jul 14 18:06:29.850731 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:29.850558 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e6ecbdb42ef29f3412c7581f4add17629e23e25579ad7c394e06b940d8469596" Jul 14 18:06:36.116103 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:36.116066 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e8afdc-894846795-6wfk2"] Jul 14 18:06:36.116574 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:36.116317 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8afdc-894846795-6wfk2" podUID="738a4ecb-925e-4159-a8fc-02f1961e1f03" containerName="maas-api" containerID="cri-o://3913c0301899231f07f140863b6e817215bf90e0ca46e1d0c736e3c67969f32c" gracePeriod=30 Jul 14 18:06:36.362765 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:36.362739 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8afdc-894846795-6wfk2" Jul 14 18:06:36.489000 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:36.488898 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/738a4ecb-925e-4159-a8fc-02f1961e1f03-openshift-service-ca\") pod \"738a4ecb-925e-4159-a8fc-02f1961e1f03\" (UID: \"738a4ecb-925e-4159-a8fc-02f1961e1f03\") " Jul 14 18:06:36.489000 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:36.488970 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vs47t\" (UniqueName: \"kubernetes.io/projected/738a4ecb-925e-4159-a8fc-02f1961e1f03-kube-api-access-vs47t\") pod \"738a4ecb-925e-4159-a8fc-02f1961e1f03\" (UID: \"738a4ecb-925e-4159-a8fc-02f1961e1f03\") " Jul 14 18:06:36.489000 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:36.489009 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/738a4ecb-925e-4159-a8fc-02f1961e1f03-maas-api-tls\") pod \"738a4ecb-925e-4159-a8fc-02f1961e1f03\" (UID: \"738a4ecb-925e-4159-a8fc-02f1961e1f03\") " Jul 14 18:06:36.489399 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:36.489374 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/738a4ecb-925e-4159-a8fc-02f1961e1f03-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "738a4ecb-925e-4159-a8fc-02f1961e1f03" (UID: "738a4ecb-925e-4159-a8fc-02f1961e1f03"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 18:06:36.491107 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:36.491086 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/738a4ecb-925e-4159-a8fc-02f1961e1f03-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "738a4ecb-925e-4159-a8fc-02f1961e1f03" (UID: "738a4ecb-925e-4159-a8fc-02f1961e1f03"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 18:06:36.491200 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:36.491172 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/738a4ecb-925e-4159-a8fc-02f1961e1f03-kube-api-access-vs47t" (OuterVolumeSpecName: "kube-api-access-vs47t") pod "738a4ecb-925e-4159-a8fc-02f1961e1f03" (UID: "738a4ecb-925e-4159-a8fc-02f1961e1f03"). InnerVolumeSpecName "kube-api-access-vs47t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:06:36.590259 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:36.590197 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/738a4ecb-925e-4159-a8fc-02f1961e1f03-maas-api-tls\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:06:36.590259 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:36.590253 2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/738a4ecb-925e-4159-a8fc-02f1961e1f03-openshift-service-ca\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:06:36.590259 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:36.590266 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vs47t\" (UniqueName: \"kubernetes.io/projected/738a4ecb-925e-4159-a8fc-02f1961e1f03-kube-api-access-vs47t\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:06:36.890655 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:36.890625 2554 generic.go:356] "Generic (PLEG): container finished" podID="738a4ecb-925e-4159-a8fc-02f1961e1f03" containerID="3913c0301899231f07f140863b6e817215bf90e0ca46e1d0c736e3c67969f32c" exitCode=0 Jul 14 18:06:36.890830 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:36.890667 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8afdc-894846795-6wfk2" event={"ID":"738a4ecb-925e-4159-a8fc-02f1961e1f03","Type":"ContainerDied","Data":"3913c0301899231f07f140863b6e817215bf90e0ca46e1d0c736e3c67969f32c"} Jul 14 18:06:36.890830 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:36.890705 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8afdc-894846795-6wfk2" event={"ID":"738a4ecb-925e-4159-a8fc-02f1961e1f03","Type":"ContainerDied","Data":"494dfcc4171a127e59caef9d6e071469ae05af68aba42ccdd0ae6d31ec4fdf90"} Jul 14 18:06:36.890830 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:36.890725 2554 scope.go:117] "RemoveContainer" containerID="3913c0301899231f07f140863b6e817215bf90e0ca46e1d0c736e3c67969f32c" Jul 14 18:06:36.890830 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:36.890670 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8afdc-894846795-6wfk2" Jul 14 18:06:36.900138 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:36.900121 2554 scope.go:117] "RemoveContainer" containerID="3913c0301899231f07f140863b6e817215bf90e0ca46e1d0c736e3c67969f32c" Jul 14 18:06:36.900433 ip-10-0-132-13 kubenswrapper[2554]: E0714 18:06:36.900414 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3913c0301899231f07f140863b6e817215bf90e0ca46e1d0c736e3c67969f32c\": container with ID starting with 3913c0301899231f07f140863b6e817215bf90e0ca46e1d0c736e3c67969f32c not found: ID does not exist" containerID="3913c0301899231f07f140863b6e817215bf90e0ca46e1d0c736e3c67969f32c" Jul 14 18:06:36.900487 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:36.900441 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3913c0301899231f07f140863b6e817215bf90e0ca46e1d0c736e3c67969f32c"} err="failed to get container status \"3913c0301899231f07f140863b6e817215bf90e0ca46e1d0c736e3c67969f32c\": rpc error: code = NotFound desc = could not find container \"3913c0301899231f07f140863b6e817215bf90e0ca46e1d0c736e3c67969f32c\": container with ID starting with 3913c0301899231f07f140863b6e817215bf90e0ca46e1d0c736e3c67969f32c not found: ID does not exist" Jul 14 18:06:36.913953 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:36.913926 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e8afdc-894846795-6wfk2"] Jul 14 18:06:36.918502 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:36.918482 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e8afdc-894846795-6wfk2"] Jul 14 18:06:38.387450 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:38.387420 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="738a4ecb-925e-4159-a8fc-02f1961e1f03" path="/var/lib/kubelet/pods/738a4ecb-925e-4159-a8fc-02f1961e1f03/volumes" Jul 14 18:06:42.945818 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:42.945781 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:06:52.149993 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:52.149960 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e9d9a7-tx8tg"] Jul 14 18:06:52.150370 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:52.150306 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d08e2e3-02bf-4809-9ddf-612135c667b5" containerName="revoke-keys" Jul 14 18:06:52.150370 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:52.150318 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d08e2e3-02bf-4809-9ddf-612135c667b5" containerName="revoke-keys" Jul 14 18:06:52.150370 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:52.150330 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="738a4ecb-925e-4159-a8fc-02f1961e1f03" containerName="maas-api" Jul 14 18:06:52.150370 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:52.150335 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="738a4ecb-925e-4159-a8fc-02f1961e1f03" containerName="maas-api" Jul 14 18:06:52.150509 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:52.150407 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="738a4ecb-925e-4159-a8fc-02f1961e1f03" containerName="maas-api" Jul 14 18:06:52.150509 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:52.150417 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d08e2e3-02bf-4809-9ddf-612135c667b5" containerName="revoke-keys" Jul 14 18:06:52.154810 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:52.154791 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e9d9a7-tx8tg" Jul 14 18:06:52.157598 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:52.157567 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9rnpk\"" Jul 14 18:06:52.160483 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:52.160458 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e9d9a7-tx8tg"] Jul 14 18:06:52.327539 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:52.327509 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e3e98bf-1608-4094-9ce8-5102c9d610e0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-e9d9a7-tx8tg\" (UID: \"5e3e98bf-1608-4094-9ce8-5102c9d610e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e9d9a7-tx8tg" Jul 14 18:06:52.428600 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:52.428529 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e3e98bf-1608-4094-9ce8-5102c9d610e0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-e9d9a7-tx8tg\" (UID: \"5e3e98bf-1608-4094-9ce8-5102c9d610e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e9d9a7-tx8tg" Jul 14 18:06:52.429092 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:52.429074 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e3e98bf-1608-4094-9ce8-5102c9d610e0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-e9d9a7-tx8tg\" (UID: \"5e3e98bf-1608-4094-9ce8-5102c9d610e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e9d9a7-tx8tg" Jul 14 18:06:52.466592 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:52.466575 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e9d9a7-tx8tg" Jul 14 18:06:52.595443 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:52.595410 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e9d9a7-tx8tg"] Jul 14 18:06:52.596965 ip-10-0-132-13 kubenswrapper[2554]: W0714 18:06:52.596943 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e3e98bf_1608_4094_9ce8_5102c9d610e0.slice/crio-f5acb297f24d12a07908155f4acf9b6df88aa8bf670f888f90e67364774efbb6 WatchSource:0}: Error finding container f5acb297f24d12a07908155f4acf9b6df88aa8bf670f888f90e67364774efbb6: Status 404 returned error can't find the container with id f5acb297f24d12a07908155f4acf9b6df88aa8bf670f888f90e67364774efbb6 Jul 14 18:06:52.981509 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:52.981472 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e9d9a7-tx8tg" event={"ID":"5e3e98bf-1608-4094-9ce8-5102c9d610e0","Type":"ContainerStarted","Data":"efcfe62be95849fc2addc3669f57e4bfe93dfc74424942e3d038b606711f54e1"} Jul 14 18:06:52.981662 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:52.981529 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e9d9a7-tx8tg" event={"ID":"5e3e98bf-1608-4094-9ce8-5102c9d610e0","Type":"ContainerStarted","Data":"f5acb297f24d12a07908155f4acf9b6df88aa8bf670f888f90e67364774efbb6"} Jul 14 18:06:53.016011 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:53.015904 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e9d9a7-tx8tg" podStartSLOduration=1.015888173 podStartE2EDuration="1.015888173s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:06:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:06:53.014638734 +0000 UTC m=+2457.155887437" watchObservedRunningTime="2026-07-14 18:06:53.015888173 +0000 UTC m=+2457.157136876" Jul 14 18:06:53.988258 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:53.988198 2554 generic.go:356] "Generic (PLEG): container finished" podID="5e3e98bf-1608-4094-9ce8-5102c9d610e0" containerID="efcfe62be95849fc2addc3669f57e4bfe93dfc74424942e3d038b606711f54e1" exitCode=0 Jul 14 18:06:53.988657 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:53.988291 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e9d9a7-tx8tg" event={"ID":"5e3e98bf-1608-4094-9ce8-5102c9d610e0","Type":"ContainerDied","Data":"efcfe62be95849fc2addc3669f57e4bfe93dfc74424942e3d038b606711f54e1"} Jul 14 18:06:55.122142 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:55.122122 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e9d9a7-tx8tg" Jul 14 18:06:55.251522 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:55.251439 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e3e98bf-1608-4094-9ce8-5102c9d610e0-maas-api-service-ca\") pod \"5e3e98bf-1608-4094-9ce8-5102c9d610e0\" (UID: \"5e3e98bf-1608-4094-9ce8-5102c9d610e0\") " Jul 14 18:06:55.251782 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:55.251760 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e3e98bf-1608-4094-9ce8-5102c9d610e0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5e3e98bf-1608-4094-9ce8-5102c9d610e0" (UID: "5e3e98bf-1608-4094-9ce8-5102c9d610e0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 18:06:55.352885 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:55.352852 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e3e98bf-1608-4094-9ce8-5102c9d610e0-maas-api-service-ca\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:06:56.000423 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:56.000394 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e9d9a7-tx8tg" Jul 14 18:06:56.000627 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:56.000442 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e9d9a7-tx8tg" event={"ID":"5e3e98bf-1608-4094-9ce8-5102c9d610e0","Type":"ContainerDied","Data":"f5acb297f24d12a07908155f4acf9b6df88aa8bf670f888f90e67364774efbb6"} Jul 14 18:06:56.000627 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:06:56.000491 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f5acb297f24d12a07908155f4acf9b6df88aa8bf670f888f90e67364774efbb6" Jul 14 18:07:02.588628 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:02.588588 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e9d9a7-66695689d6-2jclj"] Jul 14 18:07:02.589103 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:02.588825 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e9d9a7-66695689d6-2jclj" podUID="01b9c566-8d5c-4fc0-9cb3-81f13958784d" containerName="maas-api" containerID="cri-o://2ee5ea81ee61a019ee15bdb8fa57021d3e86f2249c9e093a6f9ced8852f04de7" gracePeriod=30 Jul 14 18:07:02.833838 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:02.833813 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e9d9a7-66695689d6-2jclj" Jul 14 18:07:02.917821 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:02.917788 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01b9c566-8d5c-4fc0-9cb3-81f13958784d-openshift-service-ca\") pod \"01b9c566-8d5c-4fc0-9cb3-81f13958784d\" (UID: \"01b9c566-8d5c-4fc0-9cb3-81f13958784d\") " Jul 14 18:07:02.917996 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:02.917848 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01b9c566-8d5c-4fc0-9cb3-81f13958784d-maas-api-tls\") pod \"01b9c566-8d5c-4fc0-9cb3-81f13958784d\" (UID: \"01b9c566-8d5c-4fc0-9cb3-81f13958784d\") " Jul 14 18:07:02.917996 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:02.917864 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7bnck\" (UniqueName: \"kubernetes.io/projected/01b9c566-8d5c-4fc0-9cb3-81f13958784d-kube-api-access-7bnck\") pod \"01b9c566-8d5c-4fc0-9cb3-81f13958784d\" (UID: \"01b9c566-8d5c-4fc0-9cb3-81f13958784d\") " Jul 14 18:07:02.918317 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:02.918281 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01b9c566-8d5c-4fc0-9cb3-81f13958784d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "01b9c566-8d5c-4fc0-9cb3-81f13958784d" (UID: "01b9c566-8d5c-4fc0-9cb3-81f13958784d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 18:07:02.919976 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:02.919955 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01b9c566-8d5c-4fc0-9cb3-81f13958784d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "01b9c566-8d5c-4fc0-9cb3-81f13958784d" (UID: "01b9c566-8d5c-4fc0-9cb3-81f13958784d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 18:07:02.920072 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:02.919977 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01b9c566-8d5c-4fc0-9cb3-81f13958784d-kube-api-access-7bnck" (OuterVolumeSpecName: "kube-api-access-7bnck") pod "01b9c566-8d5c-4fc0-9cb3-81f13958784d" (UID: "01b9c566-8d5c-4fc0-9cb3-81f13958784d"). InnerVolumeSpecName "kube-api-access-7bnck". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:07:03.018665 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:03.018625 2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01b9c566-8d5c-4fc0-9cb3-81f13958784d-openshift-service-ca\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:07:03.018665 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:03.018659 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01b9c566-8d5c-4fc0-9cb3-81f13958784d-maas-api-tls\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:07:03.018665 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:03.018668 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7bnck\" (UniqueName: \"kubernetes.io/projected/01b9c566-8d5c-4fc0-9cb3-81f13958784d-kube-api-access-7bnck\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:07:03.039555 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:03.039522 2554 generic.go:356] "Generic (PLEG): container finished" podID="01b9c566-8d5c-4fc0-9cb3-81f13958784d" containerID="2ee5ea81ee61a019ee15bdb8fa57021d3e86f2249c9e093a6f9ced8852f04de7" exitCode=0 Jul 14 18:07:03.039731 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:03.039565 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e9d9a7-66695689d6-2jclj" Jul 14 18:07:03.039731 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:03.039618 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e9d9a7-66695689d6-2jclj" event={"ID":"01b9c566-8d5c-4fc0-9cb3-81f13958784d","Type":"ContainerDied","Data":"2ee5ea81ee61a019ee15bdb8fa57021d3e86f2249c9e093a6f9ced8852f04de7"} Jul 14 18:07:03.039731 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:03.039680 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e9d9a7-66695689d6-2jclj" event={"ID":"01b9c566-8d5c-4fc0-9cb3-81f13958784d","Type":"ContainerDied","Data":"e4cc77be90912446f14815210bcd973517f61fc79754ff9aa91a46426086bc69"} Jul 14 18:07:03.039731 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:03.039711 2554 scope.go:117] "RemoveContainer" containerID="2ee5ea81ee61a019ee15bdb8fa57021d3e86f2249c9e093a6f9ced8852f04de7" Jul 14 18:07:03.049134 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:03.049109 2554 scope.go:117] "RemoveContainer" containerID="2ee5ea81ee61a019ee15bdb8fa57021d3e86f2249c9e093a6f9ced8852f04de7" Jul 14 18:07:03.049427 ip-10-0-132-13 kubenswrapper[2554]: E0714 18:07:03.049410 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ee5ea81ee61a019ee15bdb8fa57021d3e86f2249c9e093a6f9ced8852f04de7\": container with ID starting with 2ee5ea81ee61a019ee15bdb8fa57021d3e86f2249c9e093a6f9ced8852f04de7 not found: ID does not exist" containerID="2ee5ea81ee61a019ee15bdb8fa57021d3e86f2249c9e093a6f9ced8852f04de7" Jul 14 18:07:03.049532 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:03.049436 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ee5ea81ee61a019ee15bdb8fa57021d3e86f2249c9e093a6f9ced8852f04de7"} err="failed to get container status \"2ee5ea81ee61a019ee15bdb8fa57021d3e86f2249c9e093a6f9ced8852f04de7\": rpc error: code = NotFound desc = could not find container \"2ee5ea81ee61a019ee15bdb8fa57021d3e86f2249c9e093a6f9ced8852f04de7\": container with ID starting with 2ee5ea81ee61a019ee15bdb8fa57021d3e86f2249c9e093a6f9ced8852f04de7 not found: ID does not exist" Jul 14 18:07:03.063198 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:03.063163 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e9d9a7-66695689d6-2jclj"] Jul 14 18:07:03.068384 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:03.068355 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e9d9a7-66695689d6-2jclj"] Jul 14 18:07:04.387405 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:04.387372 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01b9c566-8d5c-4fc0-9cb3-81f13958784d" path="/var/lib/kubelet/pods/01b9c566-8d5c-4fc0-9cb3-81f13958784d/volumes" Jul 14 18:07:05.860564 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:05.860530 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:07:22.115056 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:22.115014 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8bfa74-7b6d76dd57-nkl67"] Jul 14 18:07:22.115559 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:22.115539 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01b9c566-8d5c-4fc0-9cb3-81f13958784d" containerName="maas-api" Jul 14 18:07:22.115610 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:22.115563 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="01b9c566-8d5c-4fc0-9cb3-81f13958784d" containerName="maas-api" Jul 14 18:07:22.115610 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:22.115584 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e3e98bf-1608-4094-9ce8-5102c9d610e0" containerName="revoke-keys" Jul 14 18:07:22.115610 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:22.115592 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e3e98bf-1608-4094-9ce8-5102c9d610e0" containerName="revoke-keys" Jul 14 18:07:22.115707 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:22.115699 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e3e98bf-1608-4094-9ce8-5102c9d610e0" containerName="revoke-keys" Jul 14 18:07:22.115746 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:22.115719 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="01b9c566-8d5c-4fc0-9cb3-81f13958784d" containerName="maas-api" Jul 14 18:07:22.123256 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:22.123236 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bfa74-7b6d76dd57-nkl67" Jul 14 18:07:22.126188 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:22.126168 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 18:07:22.126469 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:22.126425 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-8bfa74-serving-cert\"" Jul 14 18:07:22.126687 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:22.126671 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-ggvwm\"" Jul 14 18:07:22.127601 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:22.127584 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 14 18:07:22.132023 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:22.132002 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8bfa74-7b6d76dd57-nkl67"] Jul 14 18:07:22.171824 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:22.171793 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81fed720-d0a3-46ab-86eb-eaf55524c2fd-openshift-service-ca\") pod \"maas-api-e2e-shared-a-8bfa74-7b6d76dd57-nkl67\" (UID: \"81fed720-d0a3-46ab-86eb-eaf55524c2fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bfa74-7b6d76dd57-nkl67" Jul 14 18:07:22.171982 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:22.171831 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ncbv\" (UniqueName: \"kubernetes.io/projected/81fed720-d0a3-46ab-86eb-eaf55524c2fd-kube-api-access-4ncbv\") pod \"maas-api-e2e-shared-a-8bfa74-7b6d76dd57-nkl67\" (UID: \"81fed720-d0a3-46ab-86eb-eaf55524c2fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bfa74-7b6d76dd57-nkl67" Jul 14 18:07:22.171982 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:22.171917 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81fed720-d0a3-46ab-86eb-eaf55524c2fd-maas-api-tls\") pod \"maas-api-e2e-shared-a-8bfa74-7b6d76dd57-nkl67\" (UID: \"81fed720-d0a3-46ab-86eb-eaf55524c2fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bfa74-7b6d76dd57-nkl67" Jul 14 18:07:22.272863 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:22.272823 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81fed720-d0a3-46ab-86eb-eaf55524c2fd-maas-api-tls\") pod \"maas-api-e2e-shared-a-8bfa74-7b6d76dd57-nkl67\" (UID: \"81fed720-d0a3-46ab-86eb-eaf55524c2fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bfa74-7b6d76dd57-nkl67" Jul 14 18:07:22.273051 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:22.272885 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81fed720-d0a3-46ab-86eb-eaf55524c2fd-openshift-service-ca\") pod \"maas-api-e2e-shared-a-8bfa74-7b6d76dd57-nkl67\" (UID: \"81fed720-d0a3-46ab-86eb-eaf55524c2fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bfa74-7b6d76dd57-nkl67" Jul 14 18:07:22.273051 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:22.272910 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ncbv\" (UniqueName: \"kubernetes.io/projected/81fed720-d0a3-46ab-86eb-eaf55524c2fd-kube-api-access-4ncbv\") pod \"maas-api-e2e-shared-a-8bfa74-7b6d76dd57-nkl67\" (UID: \"81fed720-d0a3-46ab-86eb-eaf55524c2fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bfa74-7b6d76dd57-nkl67" Jul 14 18:07:22.273051 ip-10-0-132-13 kubenswrapper[2554]: E0714 18:07:22.272982 2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-8bfa74-serving-cert: secret "maas-api-e2e-shared-a-8bfa74-serving-cert" not found Jul 14 18:07:22.273182 ip-10-0-132-13 kubenswrapper[2554]: E0714 18:07:22.273060 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81fed720-d0a3-46ab-86eb-eaf55524c2fd-maas-api-tls podName:81fed720-d0a3-46ab-86eb-eaf55524c2fd nodeName:}" failed. No retries permitted until 2026-07-14 18:07:22.773041788 +0000 UTC m=+2486.914290469 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/81fed720-d0a3-46ab-86eb-eaf55524c2fd-maas-api-tls") pod "maas-api-e2e-shared-a-8bfa74-7b6d76dd57-nkl67" (UID: "81fed720-d0a3-46ab-86eb-eaf55524c2fd") : secret "maas-api-e2e-shared-a-8bfa74-serving-cert" not found Jul 14 18:07:22.273529 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:22.273511 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81fed720-d0a3-46ab-86eb-eaf55524c2fd-openshift-service-ca\") pod \"maas-api-e2e-shared-a-8bfa74-7b6d76dd57-nkl67\" (UID: \"81fed720-d0a3-46ab-86eb-eaf55524c2fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bfa74-7b6d76dd57-nkl67" Jul 14 18:07:22.284140 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:22.284117 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ncbv\" (UniqueName: \"kubernetes.io/projected/81fed720-d0a3-46ab-86eb-eaf55524c2fd-kube-api-access-4ncbv\") pod \"maas-api-e2e-shared-a-8bfa74-7b6d76dd57-nkl67\" (UID: \"81fed720-d0a3-46ab-86eb-eaf55524c2fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bfa74-7b6d76dd57-nkl67" Jul 14 18:07:22.777294 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:22.777250 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81fed720-d0a3-46ab-86eb-eaf55524c2fd-maas-api-tls\") pod \"maas-api-e2e-shared-a-8bfa74-7b6d76dd57-nkl67\" (UID: \"81fed720-d0a3-46ab-86eb-eaf55524c2fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bfa74-7b6d76dd57-nkl67" Jul 14 18:07:22.779630 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:22.779612 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81fed720-d0a3-46ab-86eb-eaf55524c2fd-maas-api-tls\") pod \"maas-api-e2e-shared-a-8bfa74-7b6d76dd57-nkl67\" (UID: \"81fed720-d0a3-46ab-86eb-eaf55524c2fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bfa74-7b6d76dd57-nkl67" Jul 14 18:07:23.035786 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:23.035683 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bfa74-7b6d76dd57-nkl67" Jul 14 18:07:23.167409 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:23.167374 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8bfa74-7b6d76dd57-nkl67"] Jul 14 18:07:23.168780 ip-10-0-132-13 kubenswrapper[2554]: W0714 18:07:23.168755 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81fed720_d0a3_46ab_86eb_eaf55524c2fd.slice/crio-df9d9b65aeefa575818a318e859769ff7d74fdef8a0328027b815e0eb166e1b5 WatchSource:0}: Error finding container df9d9b65aeefa575818a318e859769ff7d74fdef8a0328027b815e0eb166e1b5: Status 404 returned error can't find the container with id df9d9b65aeefa575818a318e859769ff7d74fdef8a0328027b815e0eb166e1b5 Jul 14 18:07:24.157569 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:24.157532 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bfa74-7b6d76dd57-nkl67" event={"ID":"81fed720-d0a3-46ab-86eb-eaf55524c2fd","Type":"ContainerStarted","Data":"21ed4ea96a44e0c86f839bf6d66d1fc7937d39fb9e5844bffc725749e0b6311d"} Jul 14 18:07:24.157744 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:24.157697 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bfa74-7b6d76dd57-nkl67" Jul 14 18:07:24.157744 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:24.157738 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bfa74-7b6d76dd57-nkl67" event={"ID":"81fed720-d0a3-46ab-86eb-eaf55524c2fd","Type":"ContainerStarted","Data":"df9d9b65aeefa575818a318e859769ff7d74fdef8a0328027b815e0eb166e1b5"} Jul 14 18:07:24.181424 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:24.181362 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bfa74-7b6d76dd57-nkl67" podStartSLOduration=1.305663619 podStartE2EDuration="2.181342717s" totalImagesPullingTime="875.679098ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:07:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:07:24.177619892 +0000 UTC m=+2488.318868597" watchObservedRunningTime="2026-07-14 18:07:24.181342717 +0000 UTC m=+2488.322591422" Jul 14 18:07:26.956434 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:26.956400 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:07:30.096177 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:30.096142 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-4f6333-594b47c87c-7z49f"] Jul 14 18:07:30.099316 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:30.099297 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f6333-594b47c87c-7z49f" Jul 14 18:07:30.102590 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:30.102570 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-4f6333-serving-cert\"" Jul 14 18:07:30.115054 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:30.115028 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-4f6333-594b47c87c-7z49f"] Jul 14 18:07:30.142995 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:30.142965 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfb6a4b7-6c11-476d-ae0a-e51f953de711-openshift-service-ca\") pod \"maas-api-e2e-shared-b-4f6333-594b47c87c-7z49f\" (UID: \"bfb6a4b7-6c11-476d-ae0a-e51f953de711\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f6333-594b47c87c-7z49f" Jul 14 18:07:30.142995 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:30.142998 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2dn7\" (UniqueName: \"kubernetes.io/projected/bfb6a4b7-6c11-476d-ae0a-e51f953de711-kube-api-access-r2dn7\") pod \"maas-api-e2e-shared-b-4f6333-594b47c87c-7z49f\" (UID: \"bfb6a4b7-6c11-476d-ae0a-e51f953de711\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f6333-594b47c87c-7z49f" Jul 14 18:07:30.143167 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:30.143029 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bfb6a4b7-6c11-476d-ae0a-e51f953de711-maas-api-tls\") pod \"maas-api-e2e-shared-b-4f6333-594b47c87c-7z49f\" (UID: \"bfb6a4b7-6c11-476d-ae0a-e51f953de711\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f6333-594b47c87c-7z49f" Jul 14 18:07:30.168765 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:30.168736 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bfa74-7b6d76dd57-nkl67" Jul 14 18:07:30.243891 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:30.243852 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfb6a4b7-6c11-476d-ae0a-e51f953de711-openshift-service-ca\") pod \"maas-api-e2e-shared-b-4f6333-594b47c87c-7z49f\" (UID: \"bfb6a4b7-6c11-476d-ae0a-e51f953de711\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f6333-594b47c87c-7z49f" Jul 14 18:07:30.243891 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:30.243889 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2dn7\" (UniqueName: \"kubernetes.io/projected/bfb6a4b7-6c11-476d-ae0a-e51f953de711-kube-api-access-r2dn7\") pod \"maas-api-e2e-shared-b-4f6333-594b47c87c-7z49f\" (UID: \"bfb6a4b7-6c11-476d-ae0a-e51f953de711\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f6333-594b47c87c-7z49f" Jul 14 18:07:30.244149 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:30.244041 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bfb6a4b7-6c11-476d-ae0a-e51f953de711-maas-api-tls\") pod \"maas-api-e2e-shared-b-4f6333-594b47c87c-7z49f\" (UID: \"bfb6a4b7-6c11-476d-ae0a-e51f953de711\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f6333-594b47c87c-7z49f" Jul 14 18:07:30.244269 ip-10-0-132-13 kubenswrapper[2554]: E0714 18:07:30.244182 2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-4f6333-serving-cert: secret "maas-api-e2e-shared-b-4f6333-serving-cert" not found Jul 14 18:07:30.244355 ip-10-0-132-13 kubenswrapper[2554]: E0714 18:07:30.244273 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfb6a4b7-6c11-476d-ae0a-e51f953de711-maas-api-tls podName:bfb6a4b7-6c11-476d-ae0a-e51f953de711 nodeName:}" failed. No retries permitted until 2026-07-14 18:07:30.744256941 +0000 UTC m=+2494.885505623 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bfb6a4b7-6c11-476d-ae0a-e51f953de711-maas-api-tls") pod "maas-api-e2e-shared-b-4f6333-594b47c87c-7z49f" (UID: "bfb6a4b7-6c11-476d-ae0a-e51f953de711") : secret "maas-api-e2e-shared-b-4f6333-serving-cert" not found Jul 14 18:07:30.244553 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:30.244533 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfb6a4b7-6c11-476d-ae0a-e51f953de711-openshift-service-ca\") pod \"maas-api-e2e-shared-b-4f6333-594b47c87c-7z49f\" (UID: \"bfb6a4b7-6c11-476d-ae0a-e51f953de711\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f6333-594b47c87c-7z49f" Jul 14 18:07:30.256292 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:30.256269 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2dn7\" (UniqueName: \"kubernetes.io/projected/bfb6a4b7-6c11-476d-ae0a-e51f953de711-kube-api-access-r2dn7\") pod \"maas-api-e2e-shared-b-4f6333-594b47c87c-7z49f\" (UID: \"bfb6a4b7-6c11-476d-ae0a-e51f953de711\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f6333-594b47c87c-7z49f" Jul 14 18:07:30.748129 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:30.748093 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bfb6a4b7-6c11-476d-ae0a-e51f953de711-maas-api-tls\") pod \"maas-api-e2e-shared-b-4f6333-594b47c87c-7z49f\" (UID: \"bfb6a4b7-6c11-476d-ae0a-e51f953de711\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f6333-594b47c87c-7z49f" Jul 14 18:07:30.750450 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:30.750431 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bfb6a4b7-6c11-476d-ae0a-e51f953de711-maas-api-tls\") pod \"maas-api-e2e-shared-b-4f6333-594b47c87c-7z49f\" (UID: \"bfb6a4b7-6c11-476d-ae0a-e51f953de711\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f6333-594b47c87c-7z49f" Jul 14 18:07:31.009580 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:31.009483 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f6333-594b47c87c-7z49f" Jul 14 18:07:31.141356 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:31.141312 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-4f6333-594b47c87c-7z49f"] Jul 14 18:07:31.142847 ip-10-0-132-13 kubenswrapper[2554]: W0714 18:07:31.142818 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfb6a4b7_6c11_476d_ae0a_e51f953de711.slice/crio-2d631e9a92f6a3adbb3ac87a4442f370df16ddec608137745d7b40910709aee3 WatchSource:0}: Error finding container 2d631e9a92f6a3adbb3ac87a4442f370df16ddec608137745d7b40910709aee3: Status 404 returned error can't find the container with id 2d631e9a92f6a3adbb3ac87a4442f370df16ddec608137745d7b40910709aee3 Jul 14 18:07:31.207945 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:31.207910 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f6333-594b47c87c-7z49f" event={"ID":"bfb6a4b7-6c11-476d-ae0a-e51f953de711","Type":"ContainerStarted","Data":"2d631e9a92f6a3adbb3ac87a4442f370df16ddec608137745d7b40910709aee3"} Jul 14 18:07:32.215755 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:32.215721 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f6333-594b47c87c-7z49f" event={"ID":"bfb6a4b7-6c11-476d-ae0a-e51f953de711","Type":"ContainerStarted","Data":"9fa8a1de0a1d6b646472e1ed6f29f0510159512bc2017743b22f40b3b7e488d6"} Jul 14 18:07:32.216169 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:32.215859 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f6333-594b47c87c-7z49f" Jul 14 18:07:32.257145 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:32.257097 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f6333-594b47c87c-7z49f" podStartSLOduration=1.8881428919999999 podStartE2EDuration="2.257080645s" totalImagesPullingTime="368.937753ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:07:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:07:32.255581071 +0000 UTC m=+2496.396829776" watchObservedRunningTime="2026-07-14 18:07:32.257080645 +0000 UTC m=+2496.398329348" Jul 14 18:07:38.226455 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:38.226424 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f6333-594b47c87c-7z49f" Jul 14 18:07:42.549024 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:07:42.548984 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:08:02.741336 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:08:02.741300 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:08:14.101716 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:08:14.101679 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-4f6333/auth-test-model-4f6333-kserve-f6768c6f9-zw5z5"] Jul 14 18:08:14.105252 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:08:14.105236 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4f6333/auth-test-model-4f6333-kserve-f6768c6f9-zw5z5" Jul 14 18:08:14.108943 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:08:14.108913 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-4f6333\"/\"kube-root-ca.crt\"" Jul 14 18:08:14.109065 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:08:14.108968 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-4f6333\"/\"default-dockercfg-nwbq6\"" Jul 14 18:08:14.109065 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:08:14.108971 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-4f6333\"/\"auth-test-model-4f6333-kserve-self-signed-certs\"" Jul 14 18:08:14.109065 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:08:14.109045 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-4f6333\"/\"openshift-service-ca.crt\"" Jul 14 18:08:14.115117 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:08:14.115095 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-4f6333/auth-test-model-4f6333-kserve-f6768c6f9-zw5z5"] Jul 14 18:08:14.116331 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:08:14.116311 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7-dshm\") pod \"auth-test-model-4f6333-kserve-f6768c6f9-zw5z5\" (UID: \"b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7\") " pod="ai-tenant-e2e-shared-b-4f6333/auth-test-model-4f6333-kserve-f6768c6f9-zw5z5" Jul 14 18:08:14.116431 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:08:14.116339 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7-tmp-dir\") pod \"auth-test-model-4f6333-kserve-f6768c6f9-zw5z5\" (UID: \"b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7\") " pod="ai-tenant-e2e-shared-b-4f6333/auth-test-model-4f6333-kserve-f6768c6f9-zw5z5" Jul 14 18:08:14.116431 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:08:14.116372 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7-tls-certs\") pod \"auth-test-model-4f6333-kserve-f6768c6f9-zw5z5\" (UID: \"b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7\") " pod="ai-tenant-e2e-shared-b-4f6333/auth-test-model-4f6333-kserve-f6768c6f9-zw5z5" Jul 14 18:08:14.116544 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:08:14.116477 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qs7cw\" (UniqueName: \"kubernetes.io/projected/b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7-kube-api-access-qs7cw\") pod \"auth-test-model-4f6333-kserve-f6768c6f9-zw5z5\" (UID: \"b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7\") " pod="ai-tenant-e2e-shared-b-4f6333/auth-test-model-4f6333-kserve-f6768c6f9-zw5z5" Jul 14 18:08:14.116604 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:08:14.116568 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7-home\") pod \"auth-test-model-4f6333-kserve-f6768c6f9-zw5z5\" (UID: \"b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7\") " pod="ai-tenant-e2e-shared-b-4f6333/auth-test-model-4f6333-kserve-f6768c6f9-zw5z5" Jul 14 18:08:14.116640 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:08:14.116625 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7-model-cache\") pod \"auth-test-model-4f6333-kserve-f6768c6f9-zw5z5\" (UID: \"b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7\") " pod="ai-tenant-e2e-shared-b-4f6333/auth-test-model-4f6333-kserve-f6768c6f9-zw5z5" Jul 14 18:08:14.217645 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:08:14.217598 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7-tls-certs\") pod \"auth-test-model-4f6333-kserve-f6768c6f9-zw5z5\" (UID: \"b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7\") " pod="ai-tenant-e2e-shared-b-4f6333/auth-test-model-4f6333-kserve-f6768c6f9-zw5z5" Jul 14 18:08:14.217829 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:08:14.217657 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qs7cw\" (UniqueName: \"kubernetes.io/projected/b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7-kube-api-access-qs7cw\") pod \"auth-test-model-4f6333-kserve-f6768c6f9-zw5z5\" (UID: \"b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7\") " pod="ai-tenant-e2e-shared-b-4f6333/auth-test-model-4f6333-kserve-f6768c6f9-zw5z5" Jul 14 18:08:14.217829 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:08:14.217725 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7-home\") pod \"auth-test-model-4f6333-kserve-f6768c6f9-zw5z5\" (UID: \"b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7\") " pod="ai-tenant-e2e-shared-b-4f6333/auth-test-model-4f6333-kserve-f6768c6f9-zw5z5" Jul 14 18:08:14.217829 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:08:14.217770 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7-model-cache\") pod \"auth-test-model-4f6333-kserve-f6768c6f9-zw5z5\" (UID: \"b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7\") " pod="ai-tenant-e2e-shared-b-4f6333/auth-test-model-4f6333-kserve-f6768c6f9-zw5z5" Jul 14 18:08:14.217829 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:08:14.217805 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7-dshm\") pod \"auth-test-model-4f6333-kserve-f6768c6f9-zw5z5\" (UID: \"b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7\") " pod="ai-tenant-e2e-shared-b-4f6333/auth-test-model-4f6333-kserve-f6768c6f9-zw5z5" Jul 14 18:08:14.218078 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:08:14.217915 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7-tmp-dir\") pod \"auth-test-model-4f6333-kserve-f6768c6f9-zw5z5\" (UID: \"b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7\") " pod="ai-tenant-e2e-shared-b-4f6333/auth-test-model-4f6333-kserve-f6768c6f9-zw5z5" Jul 14 18:08:14.218254 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:08:14.218207 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7-home\") pod \"auth-test-model-4f6333-kserve-f6768c6f9-zw5z5\" (UID: \"b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7\") " pod="ai-tenant-e2e-shared-b-4f6333/auth-test-model-4f6333-kserve-f6768c6f9-zw5z5" Jul 14 18:08:14.218334 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:08:14.218278 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7-model-cache\") pod \"auth-test-model-4f6333-kserve-f6768c6f9-zw5z5\" (UID: \"b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7\") " pod="ai-tenant-e2e-shared-b-4f6333/auth-test-model-4f6333-kserve-f6768c6f9-zw5z5" Jul 14 18:08:14.218334 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:08:14.218323 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7-tmp-dir\") pod \"auth-test-model-4f6333-kserve-f6768c6f9-zw5z5\" (UID: \"b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7\") " pod="ai-tenant-e2e-shared-b-4f6333/auth-test-model-4f6333-kserve-f6768c6f9-zw5z5" Jul 14 18:08:14.220050 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:08:14.220026 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7-dshm\") pod \"auth-test-model-4f6333-kserve-f6768c6f9-zw5z5\" (UID: \"b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7\") " pod="ai-tenant-e2e-shared-b-4f6333/auth-test-model-4f6333-kserve-f6768c6f9-zw5z5" Jul 14 18:08:14.220308 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:08:14.220292 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7-tls-certs\") pod \"auth-test-model-4f6333-kserve-f6768c6f9-zw5z5\" (UID: \"b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7\") " pod="ai-tenant-e2e-shared-b-4f6333/auth-test-model-4f6333-kserve-f6768c6f9-zw5z5" Jul 14 18:08:14.226650 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:08:14.226633 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qs7cw\" (UniqueName: \"kubernetes.io/projected/b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7-kube-api-access-qs7cw\") pod \"auth-test-model-4f6333-kserve-f6768c6f9-zw5z5\" (UID: \"b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7\") " pod="ai-tenant-e2e-shared-b-4f6333/auth-test-model-4f6333-kserve-f6768c6f9-zw5z5" Jul 14 18:08:14.414400 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:08:14.414308 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4f6333/auth-test-model-4f6333-kserve-f6768c6f9-zw5z5" Jul 14 18:08:14.548847 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:08:14.548819 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-4f6333/auth-test-model-4f6333-kserve-f6768c6f9-zw5z5"] Jul 14 18:08:14.551550 ip-10-0-132-13 kubenswrapper[2554]: W0714 18:08:14.551515 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb3f303bd_48c5_4e1b_aad3_e0b2f36c8ce7.slice/crio-f8a01bf81808af2c7b1ce6d518f324647d21a520395edc1ab839510b6576c011 WatchSource:0}: Error finding container f8a01bf81808af2c7b1ce6d518f324647d21a520395edc1ab839510b6576c011: Status 404 returned error can't find the container with id f8a01bf81808af2c7b1ce6d518f324647d21a520395edc1ab839510b6576c011 Jul 14 18:08:15.479899 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:08:15.479861 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4f6333/auth-test-model-4f6333-kserve-f6768c6f9-zw5z5" event={"ID":"b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7","Type":"ContainerStarted","Data":"60bacf3448397b312034cb9d31895b14e569027f8d6843cb744143bd58cc66c8"} Jul 14 18:08:15.480303 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:08:15.479913 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4f6333/auth-test-model-4f6333-kserve-f6768c6f9-zw5z5" event={"ID":"b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7","Type":"ContainerStarted","Data":"f8a01bf81808af2c7b1ce6d518f324647d21a520395edc1ab839510b6576c011"} Jul 14 18:08:15.497290 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:08:15.497242 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-4f6333/auth-test-model-4f6333-kserve-f6768c6f9-zw5z5" podStartSLOduration=1.497227826 podStartE2EDuration="1.497227826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:08:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:08:15.496441024 +0000 UTC m=+2539.637689728" watchObservedRunningTime="2026-07-14 18:08:15.497227826 +0000 UTC m=+2539.638476538" Jul 14 18:08:19.543638 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:08:19.543598 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:08:24.415379 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:08:24.415332 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-4f6333/auth-test-model-4f6333-kserve-f6768c6f9-zw5z5" Jul 14 18:08:24.415379 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:08:24.415387 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-4f6333/auth-test-model-4f6333-kserve-f6768c6f9-zw5z5" Jul 14 18:08:24.427937 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:08:24.427898 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-4f6333/auth-test-model-4f6333-kserve-f6768c6f9-zw5z5" Jul 14 18:08:24.540831 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:08:24.540806 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-4f6333/auth-test-model-4f6333-kserve-f6768c6f9-zw5z5" Jul 14 18:08:33.150293 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:08:33.150243 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:08:51.253151 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:08:51.253116 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:09:07.146387 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:09:07.146350 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:09:25.847696 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:09:25.847617 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:09:43.447019 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:09:43.446983 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:09:56.347977 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:09:56.347945 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:10:10.266532 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:10:10.266491 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-4f6333/auth-test-model-4f6333-kserve-f6768c6f9-zw5z5"] Jul 14 18:10:12.840741 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:10:12.840702 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:10:25.286599 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:10:25.286561 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-4f6333/auth-test-model-4f6333-kserve-f6768c6f9-zw5z5" podUID="b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7" containerName="main" containerID="cri-o://60bacf3448397b312034cb9d31895b14e569027f8d6843cb744143bd58cc66c8" gracePeriod=45 Jul 14 18:10:25.531492 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:10:25.531466 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4f6333/auth-test-model-4f6333-kserve-f6768c6f9-zw5z5" Jul 14 18:10:25.651913 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:10:25.651879 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7-dshm\") pod \"b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7\" (UID: \"b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7\") " Jul 14 18:10:25.652094 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:10:25.651924 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qs7cw\" (UniqueName: \"kubernetes.io/projected/b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7-kube-api-access-qs7cw\") pod \"b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7\" (UID: \"b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7\") " Jul 14 18:10:25.652094 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:10:25.651954 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7-home\") pod \"b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7\" (UID: \"b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7\") " Jul 14 18:10:25.652094 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:10:25.651980 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7-tmp-dir\") pod \"b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7\" (UID: \"b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7\") " Jul 14 18:10:25.652094 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:10:25.652031 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7-tls-certs\") pod \"b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7\" (UID: \"b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7\") " Jul 14 18:10:25.652094 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:10:25.652055 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7-model-cache\") pod \"b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7\" (UID: \"b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7\") " Jul 14 18:10:25.652421 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:10:25.652284 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7-home" (OuterVolumeSpecName: "home") pod "b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7" (UID: "b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:10:25.652421 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:10:25.652304 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7" (UID: "b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:10:25.652421 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:10:25.652353 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7-model-cache" (OuterVolumeSpecName: "model-cache") pod "b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7" (UID: "b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:10:25.654096 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:10:25.654075 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7-dshm" (OuterVolumeSpecName: "dshm") pod "b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7" (UID: "b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:10:25.654163 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:10:25.654118 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7-kube-api-access-qs7cw" (OuterVolumeSpecName: "kube-api-access-qs7cw") pod "b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7" (UID: "b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7"). InnerVolumeSpecName "kube-api-access-qs7cw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:10:25.654202 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:10:25.654179 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7" (UID: "b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 18:10:25.752823 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:10:25.752792 2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7-model-cache\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:10:25.752823 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:10:25.752816 2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7-dshm\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:10:25.752823 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:10:25.752825 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qs7cw\" (UniqueName: \"kubernetes.io/projected/b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7-kube-api-access-qs7cw\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:10:25.753022 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:10:25.752834 2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7-home\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:10:25.753022 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:10:25.752843 2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7-tmp-dir\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:10:25.753022 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:10:25.752849 2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7-tls-certs\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:10:26.206862 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:10:26.206825 2554 generic.go:356] "Generic (PLEG): container finished" podID="b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7" containerID="60bacf3448397b312034cb9d31895b14e569027f8d6843cb744143bd58cc66c8" exitCode=0 Jul 14 18:10:26.207037 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:10:26.206870 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4f6333/auth-test-model-4f6333-kserve-f6768c6f9-zw5z5" Jul 14 18:10:26.207037 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:10:26.206912 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4f6333/auth-test-model-4f6333-kserve-f6768c6f9-zw5z5" event={"ID":"b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7","Type":"ContainerDied","Data":"60bacf3448397b312034cb9d31895b14e569027f8d6843cb744143bd58cc66c8"} Jul 14 18:10:26.207037 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:10:26.206959 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4f6333/auth-test-model-4f6333-kserve-f6768c6f9-zw5z5" event={"ID":"b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7","Type":"ContainerDied","Data":"f8a01bf81808af2c7b1ce6d518f324647d21a520395edc1ab839510b6576c011"} Jul 14 18:10:26.207037 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:10:26.206979 2554 scope.go:117] "RemoveContainer" containerID="60bacf3448397b312034cb9d31895b14e569027f8d6843cb744143bd58cc66c8" Jul 14 18:10:26.220715 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:10:26.220691 2554 scope.go:117] "RemoveContainer" containerID="60bacf3448397b312034cb9d31895b14e569027f8d6843cb744143bd58cc66c8" Jul 14 18:10:26.220976 ip-10-0-132-13 kubenswrapper[2554]: E0714 18:10:26.220956 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60bacf3448397b312034cb9d31895b14e569027f8d6843cb744143bd58cc66c8\": container with ID starting with 60bacf3448397b312034cb9d31895b14e569027f8d6843cb744143bd58cc66c8 not found: ID does not exist" containerID="60bacf3448397b312034cb9d31895b14e569027f8d6843cb744143bd58cc66c8" Jul 14 18:10:26.221023 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:10:26.220984 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60bacf3448397b312034cb9d31895b14e569027f8d6843cb744143bd58cc66c8"} err="failed to get container status \"60bacf3448397b312034cb9d31895b14e569027f8d6843cb744143bd58cc66c8\": rpc error: code = NotFound desc = could not find container \"60bacf3448397b312034cb9d31895b14e569027f8d6843cb744143bd58cc66c8\": container with ID starting with 60bacf3448397b312034cb9d31895b14e569027f8d6843cb744143bd58cc66c8 not found: ID does not exist" Jul 14 18:10:26.228729 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:10:26.228702 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-4f6333/auth-test-model-4f6333-kserve-f6768c6f9-zw5z5"] Jul 14 18:10:26.232810 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:10:26.232786 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-4f6333/auth-test-model-4f6333-kserve-f6768c6f9-zw5z5"] Jul 14 18:10:26.388194 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:10:26.388164 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7" path="/var/lib/kubelet/pods/b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7/volumes" Jul 14 18:10:36.143463 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:10:36.143428 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:10:46.855023 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:10:46.854988 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:10:56.505299 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:10:56.505232 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pk9h2_f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726/ovn-acl-logging/0.log" Jul 14 18:10:56.514301 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:10:56.514280 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pk9h2_f3f90a0c-1b3b-415f-9ed6-84bcbdd6a726/ovn-acl-logging/0.log" Jul 14 18:11:05.851967 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:05.851936 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:11:25.254049 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:25.254002 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:11:43.248231 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:43.248181 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:11:47.076288 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:47.076254 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-8bfa74/rate-test-model-8bfa74-kserve-9cfd4d787-h59r4"] Jul 14 18:11:47.076661 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:47.076647 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7" containerName="main" Jul 14 18:11:47.076709 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:47.076662 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7" containerName="main" Jul 14 18:11:47.076751 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:47.076742 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3f303bd-48c5-4e1b-aad3-e0b2f36c8ce7" containerName="main" Jul 14 18:11:47.079860 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:47.079838 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8bfa74/rate-test-model-8bfa74-kserve-9cfd4d787-h59r4" Jul 14 18:11:47.083955 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:47.083932 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-8bfa74\"/\"openshift-service-ca.crt\"" Jul 14 18:11:47.084084 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:47.083982 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-8bfa74\"/\"kube-root-ca.crt\"" Jul 14 18:11:47.084084 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:47.083994 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-8bfa74\"/\"rate-test-model-8bfa74-kserve-self-signed-certs\"" Jul 14 18:11:47.084084 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:47.084014 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-8bfa74\"/\"default-dockercfg-56rwl\"" Jul 14 18:11:47.089133 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:47.089114 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8bfa74/rate-test-model-8bfa74-kserve-9cfd4d787-h59r4"] Jul 14 18:11:47.220286 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:47.220254 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59bdcbba-21c6-431f-9ecb-bbd372af4aef-tls-certs\") pod \"rate-test-model-8bfa74-kserve-9cfd4d787-h59r4\" (UID: \"59bdcbba-21c6-431f-9ecb-bbd372af4aef\") " pod="ai-tenant-e2e-shared-a-8bfa74/rate-test-model-8bfa74-kserve-9cfd4d787-h59r4" Jul 14 18:11:47.220465 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:47.220296 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdj9s\" (UniqueName: \"kubernetes.io/projected/59bdcbba-21c6-431f-9ecb-bbd372af4aef-kube-api-access-rdj9s\") pod \"rate-test-model-8bfa74-kserve-9cfd4d787-h59r4\" (UID: \"59bdcbba-21c6-431f-9ecb-bbd372af4aef\") " pod="ai-tenant-e2e-shared-a-8bfa74/rate-test-model-8bfa74-kserve-9cfd4d787-h59r4" Jul 14 18:11:47.220465 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:47.220324 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59bdcbba-21c6-431f-9ecb-bbd372af4aef-home\") pod \"rate-test-model-8bfa74-kserve-9cfd4d787-h59r4\" (UID: \"59bdcbba-21c6-431f-9ecb-bbd372af4aef\") " pod="ai-tenant-e2e-shared-a-8bfa74/rate-test-model-8bfa74-kserve-9cfd4d787-h59r4" Jul 14 18:11:47.220465 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:47.220405 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59bdcbba-21c6-431f-9ecb-bbd372af4aef-tmp-dir\") pod \"rate-test-model-8bfa74-kserve-9cfd4d787-h59r4\" (UID: \"59bdcbba-21c6-431f-9ecb-bbd372af4aef\") " pod="ai-tenant-e2e-shared-a-8bfa74/rate-test-model-8bfa74-kserve-9cfd4d787-h59r4" Jul 14 18:11:47.220465 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:47.220431 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59bdcbba-21c6-431f-9ecb-bbd372af4aef-model-cache\") pod \"rate-test-model-8bfa74-kserve-9cfd4d787-h59r4\" (UID: \"59bdcbba-21c6-431f-9ecb-bbd372af4aef\") " pod="ai-tenant-e2e-shared-a-8bfa74/rate-test-model-8bfa74-kserve-9cfd4d787-h59r4" Jul 14 18:11:47.220631 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:47.220469 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59bdcbba-21c6-431f-9ecb-bbd372af4aef-dshm\") pod \"rate-test-model-8bfa74-kserve-9cfd4d787-h59r4\" (UID: \"59bdcbba-21c6-431f-9ecb-bbd372af4aef\") " pod="ai-tenant-e2e-shared-a-8bfa74/rate-test-model-8bfa74-kserve-9cfd4d787-h59r4" Jul 14 18:11:47.320901 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:47.320858 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59bdcbba-21c6-431f-9ecb-bbd372af4aef-dshm\") pod \"rate-test-model-8bfa74-kserve-9cfd4d787-h59r4\" (UID: \"59bdcbba-21c6-431f-9ecb-bbd372af4aef\") " pod="ai-tenant-e2e-shared-a-8bfa74/rate-test-model-8bfa74-kserve-9cfd4d787-h59r4" Jul 14 18:11:47.321116 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:47.320922 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59bdcbba-21c6-431f-9ecb-bbd372af4aef-tls-certs\") pod \"rate-test-model-8bfa74-kserve-9cfd4d787-h59r4\" (UID: \"59bdcbba-21c6-431f-9ecb-bbd372af4aef\") " pod="ai-tenant-e2e-shared-a-8bfa74/rate-test-model-8bfa74-kserve-9cfd4d787-h59r4" Jul 14 18:11:47.321116 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:47.320946 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdj9s\" (UniqueName: \"kubernetes.io/projected/59bdcbba-21c6-431f-9ecb-bbd372af4aef-kube-api-access-rdj9s\") pod \"rate-test-model-8bfa74-kserve-9cfd4d787-h59r4\" (UID: \"59bdcbba-21c6-431f-9ecb-bbd372af4aef\") " pod="ai-tenant-e2e-shared-a-8bfa74/rate-test-model-8bfa74-kserve-9cfd4d787-h59r4" Jul 14 18:11:47.321116 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:47.320973 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59bdcbba-21c6-431f-9ecb-bbd372af4aef-home\") pod \"rate-test-model-8bfa74-kserve-9cfd4d787-h59r4\" (UID: \"59bdcbba-21c6-431f-9ecb-bbd372af4aef\") " pod="ai-tenant-e2e-shared-a-8bfa74/rate-test-model-8bfa74-kserve-9cfd4d787-h59r4" Jul 14 18:11:47.321116 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:47.321000 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59bdcbba-21c6-431f-9ecb-bbd372af4aef-tmp-dir\") pod \"rate-test-model-8bfa74-kserve-9cfd4d787-h59r4\" (UID: \"59bdcbba-21c6-431f-9ecb-bbd372af4aef\") " pod="ai-tenant-e2e-shared-a-8bfa74/rate-test-model-8bfa74-kserve-9cfd4d787-h59r4" Jul 14 18:11:47.321116 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:47.321018 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59bdcbba-21c6-431f-9ecb-bbd372af4aef-model-cache\") pod \"rate-test-model-8bfa74-kserve-9cfd4d787-h59r4\" (UID: \"59bdcbba-21c6-431f-9ecb-bbd372af4aef\") " pod="ai-tenant-e2e-shared-a-8bfa74/rate-test-model-8bfa74-kserve-9cfd4d787-h59r4" Jul 14 18:11:47.321404 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:47.321362 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59bdcbba-21c6-431f-9ecb-bbd372af4aef-model-cache\") pod \"rate-test-model-8bfa74-kserve-9cfd4d787-h59r4\" (UID: \"59bdcbba-21c6-431f-9ecb-bbd372af4aef\") " pod="ai-tenant-e2e-shared-a-8bfa74/rate-test-model-8bfa74-kserve-9cfd4d787-h59r4" Jul 14 18:11:47.321472 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:47.321449 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59bdcbba-21c6-431f-9ecb-bbd372af4aef-home\") pod \"rate-test-model-8bfa74-kserve-9cfd4d787-h59r4\" (UID: \"59bdcbba-21c6-431f-9ecb-bbd372af4aef\") " pod="ai-tenant-e2e-shared-a-8bfa74/rate-test-model-8bfa74-kserve-9cfd4d787-h59r4" Jul 14 18:11:47.321519 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:47.321500 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59bdcbba-21c6-431f-9ecb-bbd372af4aef-tmp-dir\") pod \"rate-test-model-8bfa74-kserve-9cfd4d787-h59r4\" (UID: \"59bdcbba-21c6-431f-9ecb-bbd372af4aef\") " pod="ai-tenant-e2e-shared-a-8bfa74/rate-test-model-8bfa74-kserve-9cfd4d787-h59r4" Jul 14 18:11:47.323205 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:47.323184 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59bdcbba-21c6-431f-9ecb-bbd372af4aef-dshm\") pod \"rate-test-model-8bfa74-kserve-9cfd4d787-h59r4\" (UID: \"59bdcbba-21c6-431f-9ecb-bbd372af4aef\") " pod="ai-tenant-e2e-shared-a-8bfa74/rate-test-model-8bfa74-kserve-9cfd4d787-h59r4" Jul 14 18:11:47.323408 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:47.323390 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59bdcbba-21c6-431f-9ecb-bbd372af4aef-tls-certs\") pod \"rate-test-model-8bfa74-kserve-9cfd4d787-h59r4\" (UID: \"59bdcbba-21c6-431f-9ecb-bbd372af4aef\") " pod="ai-tenant-e2e-shared-a-8bfa74/rate-test-model-8bfa74-kserve-9cfd4d787-h59r4" Jul 14 18:11:47.329425 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:47.329371 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdj9s\" (UniqueName: \"kubernetes.io/projected/59bdcbba-21c6-431f-9ecb-bbd372af4aef-kube-api-access-rdj9s\") pod \"rate-test-model-8bfa74-kserve-9cfd4d787-h59r4\" (UID: \"59bdcbba-21c6-431f-9ecb-bbd372af4aef\") " pod="ai-tenant-e2e-shared-a-8bfa74/rate-test-model-8bfa74-kserve-9cfd4d787-h59r4" Jul 14 18:11:47.390682 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:47.390657 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8bfa74/rate-test-model-8bfa74-kserve-9cfd4d787-h59r4" Jul 14 18:11:47.521924 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:47.521888 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8bfa74/rate-test-model-8bfa74-kserve-9cfd4d787-h59r4"] Jul 14 18:11:47.523555 ip-10-0-132-13 kubenswrapper[2554]: W0714 18:11:47.523530 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod59bdcbba_21c6_431f_9ecb_bbd372af4aef.slice/crio-e5a3c04f04805a0adb1968d1188d146d9d3a43cdb3a0ffd20ccbc7c737ceca23 WatchSource:0}: Error finding container e5a3c04f04805a0adb1968d1188d146d9d3a43cdb3a0ffd20ccbc7c737ceca23: Status 404 returned error can't find the container with id e5a3c04f04805a0adb1968d1188d146d9d3a43cdb3a0ffd20ccbc7c737ceca23 Jul 14 18:11:47.668513 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:47.668481 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8bfa74/rate-test-model-8bfa74-kserve-9cfd4d787-h59r4" event={"ID":"59bdcbba-21c6-431f-9ecb-bbd372af4aef","Type":"ContainerStarted","Data":"11b53c9a5b48fc7f195557fefd4d9ca08b74a4d34c0281b355dac35741680010"} Jul 14 18:11:47.668513 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:47.668519 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8bfa74/rate-test-model-8bfa74-kserve-9cfd4d787-h59r4" event={"ID":"59bdcbba-21c6-431f-9ecb-bbd372af4aef","Type":"ContainerStarted","Data":"e5a3c04f04805a0adb1968d1188d146d9d3a43cdb3a0ffd20ccbc7c737ceca23"} Jul 14 18:11:47.686248 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:47.686180 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-8bfa74/rate-test-model-8bfa74-kserve-9cfd4d787-h59r4" podStartSLOduration=0.68616107 podStartE2EDuration="686.16107ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:11:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:11:47.685565375 +0000 UTC m=+2751.826814079" watchObservedRunningTime="2026-07-14 18:11:47.68616107 +0000 UTC m=+2751.827409775" Jul 14 18:11:57.391192 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:57.391155 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-8bfa74/rate-test-model-8bfa74-kserve-9cfd4d787-h59r4" Jul 14 18:11:57.391710 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:57.391228 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-8bfa74/rate-test-model-8bfa74-kserve-9cfd4d787-h59r4" Jul 14 18:11:57.404096 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:57.404075 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-8bfa74/rate-test-model-8bfa74-kserve-9cfd4d787-h59r4" Jul 14 18:11:57.734283 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:57.734192 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-8bfa74/rate-test-model-8bfa74-kserve-9cfd4d787-h59r4" Jul 14 18:11:59.645703 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:11:59.645672 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:12:31.154483 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:12:31.154448 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:12:50.548134 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:12:50.548096 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:13:06.547073 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:06.547035 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:13:13.459478 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:13.459436 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8bfa74/rate-test-model-8bfa74-kserve-9cfd4d787-h59r4"] Jul 14 18:13:27.250207 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:27.250172 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:13:28.480450 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:28.480417 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-8bfa74/rate-test-model-8bfa74-kserve-9cfd4d787-h59r4" podUID="59bdcbba-21c6-431f-9ecb-bbd372af4aef" containerName="main" containerID="cri-o://11b53c9a5b48fc7f195557fefd4d9ca08b74a4d34c0281b355dac35741680010" gracePeriod=45 Jul 14 18:13:28.733052 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:28.732998 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8bfa74/rate-test-model-8bfa74-kserve-9cfd4d787-h59r4" Jul 14 18:13:28.896706 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:28.896674 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59bdcbba-21c6-431f-9ecb-bbd372af4aef-tmp-dir\") pod \"59bdcbba-21c6-431f-9ecb-bbd372af4aef\" (UID: \"59bdcbba-21c6-431f-9ecb-bbd372af4aef\") " Jul 14 18:13:28.896706 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:28.896709 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rdj9s\" (UniqueName: \"kubernetes.io/projected/59bdcbba-21c6-431f-9ecb-bbd372af4aef-kube-api-access-rdj9s\") pod \"59bdcbba-21c6-431f-9ecb-bbd372af4aef\" (UID: \"59bdcbba-21c6-431f-9ecb-bbd372af4aef\") " Jul 14 18:13:28.896942 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:28.896728 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59bdcbba-21c6-431f-9ecb-bbd372af4aef-home\") pod \"59bdcbba-21c6-431f-9ecb-bbd372af4aef\" (UID: \"59bdcbba-21c6-431f-9ecb-bbd372af4aef\") " Jul 14 18:13:28.896942 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:28.896764 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59bdcbba-21c6-431f-9ecb-bbd372af4aef-dshm\") pod \"59bdcbba-21c6-431f-9ecb-bbd372af4aef\" (UID: \"59bdcbba-21c6-431f-9ecb-bbd372af4aef\") " Jul 14 18:13:28.896942 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:28.896821 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59bdcbba-21c6-431f-9ecb-bbd372af4aef-model-cache\") pod \"59bdcbba-21c6-431f-9ecb-bbd372af4aef\" (UID: \"59bdcbba-21c6-431f-9ecb-bbd372af4aef\") " Jul 14 18:13:28.896942 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:28.896877 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59bdcbba-21c6-431f-9ecb-bbd372af4aef-tls-certs\") pod \"59bdcbba-21c6-431f-9ecb-bbd372af4aef\" (UID: \"59bdcbba-21c6-431f-9ecb-bbd372af4aef\") " Jul 14 18:13:28.897131 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:28.896988 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59bdcbba-21c6-431f-9ecb-bbd372af4aef-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "59bdcbba-21c6-431f-9ecb-bbd372af4aef" (UID: "59bdcbba-21c6-431f-9ecb-bbd372af4aef"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:13:28.897131 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:28.897080 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59bdcbba-21c6-431f-9ecb-bbd372af4aef-home" (OuterVolumeSpecName: "home") pod "59bdcbba-21c6-431f-9ecb-bbd372af4aef" (UID: "59bdcbba-21c6-431f-9ecb-bbd372af4aef"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:13:28.897131 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:28.897120 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59bdcbba-21c6-431f-9ecb-bbd372af4aef-model-cache" (OuterVolumeSpecName: "model-cache") pod "59bdcbba-21c6-431f-9ecb-bbd372af4aef" (UID: "59bdcbba-21c6-431f-9ecb-bbd372af4aef"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:13:28.897328 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:28.897133 2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59bdcbba-21c6-431f-9ecb-bbd372af4aef-tmp-dir\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:13:28.899089 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:28.899060 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59bdcbba-21c6-431f-9ecb-bbd372af4aef-dshm" (OuterVolumeSpecName: "dshm") pod "59bdcbba-21c6-431f-9ecb-bbd372af4aef" (UID: "59bdcbba-21c6-431f-9ecb-bbd372af4aef"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:13:28.899228 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:28.899156 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59bdcbba-21c6-431f-9ecb-bbd372af4aef-kube-api-access-rdj9s" (OuterVolumeSpecName: "kube-api-access-rdj9s") pod "59bdcbba-21c6-431f-9ecb-bbd372af4aef" (UID: "59bdcbba-21c6-431f-9ecb-bbd372af4aef"). InnerVolumeSpecName "kube-api-access-rdj9s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:13:28.899228 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:28.899157 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59bdcbba-21c6-431f-9ecb-bbd372af4aef-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "59bdcbba-21c6-431f-9ecb-bbd372af4aef" (UID: "59bdcbba-21c6-431f-9ecb-bbd372af4aef"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 18:13:28.998451 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:28.998383 2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59bdcbba-21c6-431f-9ecb-bbd372af4aef-tls-certs\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:13:28.998451 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:28.998408 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdj9s\" (UniqueName: \"kubernetes.io/projected/59bdcbba-21c6-431f-9ecb-bbd372af4aef-kube-api-access-rdj9s\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:13:28.998451 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:28.998421 2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59bdcbba-21c6-431f-9ecb-bbd372af4aef-home\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:13:28.998451 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:28.998433 2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59bdcbba-21c6-431f-9ecb-bbd372af4aef-dshm\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:13:28.998451 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:28.998445 2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59bdcbba-21c6-431f-9ecb-bbd372af4aef-model-cache\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:13:29.223112 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:29.223073 2554 generic.go:356] "Generic (PLEG): container finished" podID="59bdcbba-21c6-431f-9ecb-bbd372af4aef" containerID="11b53c9a5b48fc7f195557fefd4d9ca08b74a4d34c0281b355dac35741680010" exitCode=0 Jul 14 18:13:29.223293 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:29.223118 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8bfa74/rate-test-model-8bfa74-kserve-9cfd4d787-h59r4" Jul 14 18:13:29.223293 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:29.223157 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8bfa74/rate-test-model-8bfa74-kserve-9cfd4d787-h59r4" event={"ID":"59bdcbba-21c6-431f-9ecb-bbd372af4aef","Type":"ContainerDied","Data":"11b53c9a5b48fc7f195557fefd4d9ca08b74a4d34c0281b355dac35741680010"} Jul 14 18:13:29.223293 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:29.223200 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8bfa74/rate-test-model-8bfa74-kserve-9cfd4d787-h59r4" event={"ID":"59bdcbba-21c6-431f-9ecb-bbd372af4aef","Type":"ContainerDied","Data":"e5a3c04f04805a0adb1968d1188d146d9d3a43cdb3a0ffd20ccbc7c737ceca23"} Jul 14 18:13:29.223293 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:29.223240 2554 scope.go:117] "RemoveContainer" containerID="11b53c9a5b48fc7f195557fefd4d9ca08b74a4d34c0281b355dac35741680010" Jul 14 18:13:29.232690 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:29.232675 2554 scope.go:117] "RemoveContainer" containerID="11b53c9a5b48fc7f195557fefd4d9ca08b74a4d34c0281b355dac35741680010" Jul 14 18:13:29.232933 ip-10-0-132-13 kubenswrapper[2554]: E0714 18:13:29.232912 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11b53c9a5b48fc7f195557fefd4d9ca08b74a4d34c0281b355dac35741680010\": container with ID starting with 11b53c9a5b48fc7f195557fefd4d9ca08b74a4d34c0281b355dac35741680010 not found: ID does not exist" containerID="11b53c9a5b48fc7f195557fefd4d9ca08b74a4d34c0281b355dac35741680010" Jul 14 18:13:29.232988 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:29.232941 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11b53c9a5b48fc7f195557fefd4d9ca08b74a4d34c0281b355dac35741680010"} err="failed to get container status \"11b53c9a5b48fc7f195557fefd4d9ca08b74a4d34c0281b355dac35741680010\": rpc error: code = NotFound desc = could not find container \"11b53c9a5b48fc7f195557fefd4d9ca08b74a4d34c0281b355dac35741680010\": container with ID starting with 11b53c9a5b48fc7f195557fefd4d9ca08b74a4d34c0281b355dac35741680010 not found: ID does not exist" Jul 14 18:13:29.245609 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:29.245589 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8bfa74/rate-test-model-8bfa74-kserve-9cfd4d787-h59r4"] Jul 14 18:13:29.251027 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:29.250981 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-8bfa74/rate-test-model-8bfa74-kserve-9cfd4d787-h59r4"] Jul 14 18:13:30.390010 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:30.389978 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59bdcbba-21c6-431f-9ecb-bbd372af4aef" path="/var/lib/kubelet/pods/59bdcbba-21c6-431f-9ecb-bbd372af4aef/volumes" Jul 14 18:13:33.957999 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:33.957960 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8bfa74-c4hzk"] Jul 14 18:13:33.958409 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:33.958359 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59bdcbba-21c6-431f-9ecb-bbd372af4aef" containerName="main" Jul 14 18:13:33.958409 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:33.958371 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="59bdcbba-21c6-431f-9ecb-bbd372af4aef" containerName="main" Jul 14 18:13:33.958492 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:33.958435 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="59bdcbba-21c6-431f-9ecb-bbd372af4aef" containerName="main" Jul 14 18:13:33.961548 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:33.961532 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8bfa74-c4hzk" Jul 14 18:13:33.964149 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:33.964128 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9rnpk\"" Jul 14 18:13:33.968311 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:33.968276 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8bfa74-c4hzk"] Jul 14 18:13:34.140962 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:34.140932 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbd65c81-140c-49af-b200-ade9e6b5e94d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-8bfa74-c4hzk\" (UID: \"bbd65c81-140c-49af-b200-ade9e6b5e94d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8bfa74-c4hzk" Jul 14 18:13:34.241579 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:34.241489 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbd65c81-140c-49af-b200-ade9e6b5e94d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-8bfa74-c4hzk\" (UID: \"bbd65c81-140c-49af-b200-ade9e6b5e94d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8bfa74-c4hzk" Jul 14 18:13:34.242120 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:34.242099 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbd65c81-140c-49af-b200-ade9e6b5e94d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-8bfa74-c4hzk\" (UID: \"bbd65c81-140c-49af-b200-ade9e6b5e94d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8bfa74-c4hzk" Jul 14 18:13:34.274022 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:34.273999 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8bfa74-c4hzk" Jul 14 18:13:34.409182 ip-10-0-132-13 kubenswrapper[2554]: W0714 18:13:34.409144 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbd65c81_140c_49af_b200_ade9e6b5e94d.slice/crio-ec04b4f4f32380f0e136d37f1a4e82a70c48c02d8da4d2ff4de26b0b54dfcb83 WatchSource:0}: Error finding container ec04b4f4f32380f0e136d37f1a4e82a70c48c02d8da4d2ff4de26b0b54dfcb83: Status 404 returned error can't find the container with id ec04b4f4f32380f0e136d37f1a4e82a70c48c02d8da4d2ff4de26b0b54dfcb83 Jul 14 18:13:34.409424 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:34.409399 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8bfa74-c4hzk"] Jul 14 18:13:35.260668 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:35.260633 2554 generic.go:356] "Generic (PLEG): container finished" podID="bbd65c81-140c-49af-b200-ade9e6b5e94d" containerID="3cffab0a4c8270eb657dd57092c45cc8b736d68dd5b2404546d57dcb054cd840" exitCode=0 Jul 14 18:13:35.260668 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:35.260670 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8bfa74-c4hzk" event={"ID":"bbd65c81-140c-49af-b200-ade9e6b5e94d","Type":"ContainerDied","Data":"3cffab0a4c8270eb657dd57092c45cc8b736d68dd5b2404546d57dcb054cd840"} Jul 14 18:13:35.261084 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:35.260707 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8bfa74-c4hzk" event={"ID":"bbd65c81-140c-49af-b200-ade9e6b5e94d","Type":"ContainerStarted","Data":"ec04b4f4f32380f0e136d37f1a4e82a70c48c02d8da4d2ff4de26b0b54dfcb83"} Jul 14 18:13:36.397010 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:36.396989 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8bfa74-c4hzk" Jul 14 18:13:36.562492 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:36.562418 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbd65c81-140c-49af-b200-ade9e6b5e94d-maas-api-service-ca\") pod \"bbd65c81-140c-49af-b200-ade9e6b5e94d\" (UID: \"bbd65c81-140c-49af-b200-ade9e6b5e94d\") " Jul 14 18:13:36.562818 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:36.562793 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbd65c81-140c-49af-b200-ade9e6b5e94d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bbd65c81-140c-49af-b200-ade9e6b5e94d" (UID: "bbd65c81-140c-49af-b200-ade9e6b5e94d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 18:13:36.663559 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:36.663536 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbd65c81-140c-49af-b200-ade9e6b5e94d-maas-api-service-ca\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:13:37.272722 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:37.272689 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8bfa74-c4hzk" Jul 14 18:13:37.272924 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:37.272733 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8bfa74-c4hzk" event={"ID":"bbd65c81-140c-49af-b200-ade9e6b5e94d","Type":"ContainerDied","Data":"ec04b4f4f32380f0e136d37f1a4e82a70c48c02d8da4d2ff4de26b0b54dfcb83"} Jul 14 18:13:37.272924 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:37.272782 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ec04b4f4f32380f0e136d37f1a4e82a70c48c02d8da4d2ff4de26b0b54dfcb83" Jul 14 18:13:44.405340 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:44.405304 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8bfa74-7b6d76dd57-nkl67"] Jul 14 18:13:44.405791 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:44.405582 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bfa74-7b6d76dd57-nkl67" podUID="81fed720-d0a3-46ab-86eb-eaf55524c2fd" containerName="maas-api" containerID="cri-o://21ed4ea96a44e0c86f839bf6d66d1fc7937d39fb9e5844bffc725749e0b6311d" gracePeriod=30 Jul 14 18:13:44.649348 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:44.649323 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bfa74-7b6d76dd57-nkl67" Jul 14 18:13:44.834013 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:44.833927 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81fed720-d0a3-46ab-86eb-eaf55524c2fd-openshift-service-ca\") pod \"81fed720-d0a3-46ab-86eb-eaf55524c2fd\" (UID: \"81fed720-d0a3-46ab-86eb-eaf55524c2fd\") " Jul 14 18:13:44.834013 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:44.833967 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ncbv\" (UniqueName: \"kubernetes.io/projected/81fed720-d0a3-46ab-86eb-eaf55524c2fd-kube-api-access-4ncbv\") pod \"81fed720-d0a3-46ab-86eb-eaf55524c2fd\" (UID: \"81fed720-d0a3-46ab-86eb-eaf55524c2fd\") " Jul 14 18:13:44.834013 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:44.833985 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81fed720-d0a3-46ab-86eb-eaf55524c2fd-maas-api-tls\") pod \"81fed720-d0a3-46ab-86eb-eaf55524c2fd\" (UID: \"81fed720-d0a3-46ab-86eb-eaf55524c2fd\") " Jul 14 18:13:44.834321 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:44.834299 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81fed720-d0a3-46ab-86eb-eaf55524c2fd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "81fed720-d0a3-46ab-86eb-eaf55524c2fd" (UID: "81fed720-d0a3-46ab-86eb-eaf55524c2fd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 18:13:44.836098 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:44.836072 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81fed720-d0a3-46ab-86eb-eaf55524c2fd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "81fed720-d0a3-46ab-86eb-eaf55524c2fd" (UID: "81fed720-d0a3-46ab-86eb-eaf55524c2fd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 18:13:44.836236 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:44.836138 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81fed720-d0a3-46ab-86eb-eaf55524c2fd-kube-api-access-4ncbv" (OuterVolumeSpecName: "kube-api-access-4ncbv") pod "81fed720-d0a3-46ab-86eb-eaf55524c2fd" (UID: "81fed720-d0a3-46ab-86eb-eaf55524c2fd"). InnerVolumeSpecName "kube-api-access-4ncbv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:13:44.934850 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:44.934821 2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81fed720-d0a3-46ab-86eb-eaf55524c2fd-openshift-service-ca\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:13:44.934850 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:44.934845 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ncbv\" (UniqueName: \"kubernetes.io/projected/81fed720-d0a3-46ab-86eb-eaf55524c2fd-kube-api-access-4ncbv\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:13:44.934850 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:44.934855 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81fed720-d0a3-46ab-86eb-eaf55524c2fd-maas-api-tls\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:13:45.319058 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:45.319025 2554 generic.go:356] "Generic (PLEG): container finished" podID="81fed720-d0a3-46ab-86eb-eaf55524c2fd" containerID="21ed4ea96a44e0c86f839bf6d66d1fc7937d39fb9e5844bffc725749e0b6311d" exitCode=0 Jul 14 18:13:45.319240 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:45.319066 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bfa74-7b6d76dd57-nkl67" Jul 14 18:13:45.319240 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:45.319106 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bfa74-7b6d76dd57-nkl67" event={"ID":"81fed720-d0a3-46ab-86eb-eaf55524c2fd","Type":"ContainerDied","Data":"21ed4ea96a44e0c86f839bf6d66d1fc7937d39fb9e5844bffc725749e0b6311d"} Jul 14 18:13:45.319240 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:45.319154 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bfa74-7b6d76dd57-nkl67" event={"ID":"81fed720-d0a3-46ab-86eb-eaf55524c2fd","Type":"ContainerDied","Data":"df9d9b65aeefa575818a318e859769ff7d74fdef8a0328027b815e0eb166e1b5"} Jul 14 18:13:45.319240 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:45.319179 2554 scope.go:117] "RemoveContainer" containerID="21ed4ea96a44e0c86f839bf6d66d1fc7937d39fb9e5844bffc725749e0b6311d" Jul 14 18:13:45.328640 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:45.328625 2554 scope.go:117] "RemoveContainer" containerID="21ed4ea96a44e0c86f839bf6d66d1fc7937d39fb9e5844bffc725749e0b6311d" Jul 14 18:13:45.328888 ip-10-0-132-13 kubenswrapper[2554]: E0714 18:13:45.328871 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21ed4ea96a44e0c86f839bf6d66d1fc7937d39fb9e5844bffc725749e0b6311d\": container with ID starting with 21ed4ea96a44e0c86f839bf6d66d1fc7937d39fb9e5844bffc725749e0b6311d not found: ID does not exist" containerID="21ed4ea96a44e0c86f839bf6d66d1fc7937d39fb9e5844bffc725749e0b6311d" Jul 14 18:13:45.328942 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:45.328897 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21ed4ea96a44e0c86f839bf6d66d1fc7937d39fb9e5844bffc725749e0b6311d"} err="failed to get container status \"21ed4ea96a44e0c86f839bf6d66d1fc7937d39fb9e5844bffc725749e0b6311d\": rpc error: code = NotFound desc = could not find container \"21ed4ea96a44e0c86f839bf6d66d1fc7937d39fb9e5844bffc725749e0b6311d\": container with ID starting with 21ed4ea96a44e0c86f839bf6d66d1fc7937d39fb9e5844bffc725749e0b6311d not found: ID does not exist" Jul 14 18:13:45.340777 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:45.340753 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8bfa74-7b6d76dd57-nkl67"] Jul 14 18:13:45.344899 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:45.344874 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8bfa74-7b6d76dd57-nkl67"] Jul 14 18:13:46.388018 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:46.387982 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81fed720-d0a3-46ab-86eb-eaf55524c2fd" path="/var/lib/kubelet/pods/81fed720-d0a3-46ab-86eb-eaf55524c2fd/volumes" Jul 14 18:13:47.745323 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:13:47.745289 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:14:00.082111 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:00.082079 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4f6333-pp79g"] Jul 14 18:14:00.082586 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:00.082434 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81fed720-d0a3-46ab-86eb-eaf55524c2fd" containerName="maas-api" Jul 14 18:14:00.082586 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:00.082445 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="81fed720-d0a3-46ab-86eb-eaf55524c2fd" containerName="maas-api" Jul 14 18:14:00.082586 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:00.082470 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbd65c81-140c-49af-b200-ade9e6b5e94d" containerName="revoke-keys" Jul 14 18:14:00.082586 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:00.082475 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbd65c81-140c-49af-b200-ade9e6b5e94d" containerName="revoke-keys" Jul 14 18:14:00.082586 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:00.082535 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="81fed720-d0a3-46ab-86eb-eaf55524c2fd" containerName="maas-api" Jul 14 18:14:00.082586 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:00.082545 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbd65c81-140c-49af-b200-ade9e6b5e94d" containerName="revoke-keys" Jul 14 18:14:00.087262 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:00.087246 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4f6333-pp79g" Jul 14 18:14:00.089828 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:00.089808 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9rnpk\"" Jul 14 18:14:00.094722 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:00.094488 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4f6333-pp79g"] Jul 14 18:14:00.143900 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:00.143844 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c91a73c-eb8e-4396-ad08-2b0f5a9af552-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-4f6333-pp79g\" (UID: \"3c91a73c-eb8e-4396-ad08-2b0f5a9af552\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4f6333-pp79g" Jul 14 18:14:00.244684 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:00.244638 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c91a73c-eb8e-4396-ad08-2b0f5a9af552-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-4f6333-pp79g\" (UID: \"3c91a73c-eb8e-4396-ad08-2b0f5a9af552\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4f6333-pp79g" Jul 14 18:14:00.245364 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:00.245331 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c91a73c-eb8e-4396-ad08-2b0f5a9af552-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-4f6333-pp79g\" (UID: \"3c91a73c-eb8e-4396-ad08-2b0f5a9af552\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4f6333-pp79g" Jul 14 18:14:00.399264 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:00.399243 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4f6333-pp79g" Jul 14 18:14:00.538948 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:00.538917 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4f6333-pp79g"] Jul 14 18:14:00.540074 ip-10-0-132-13 kubenswrapper[2554]: W0714 18:14:00.540050 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c91a73c_eb8e_4396_ad08_2b0f5a9af552.slice/crio-276533bbf808e175df4e62c2de71508b4f452ea258ec8d82530f3622b6ba7b3b WatchSource:0}: Error finding container 276533bbf808e175df4e62c2de71508b4f452ea258ec8d82530f3622b6ba7b3b: Status 404 returned error can't find the container with id 276533bbf808e175df4e62c2de71508b4f452ea258ec8d82530f3622b6ba7b3b Jul 14 18:14:01.414675 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:01.414641 2554 generic.go:356] "Generic (PLEG): container finished" podID="3c91a73c-eb8e-4396-ad08-2b0f5a9af552" containerID="6786e67d027ee3fdad755f2265d1f2e0b989ad5916096295c9bb642dab86877e" exitCode=0 Jul 14 18:14:01.415070 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:01.414711 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4f6333-pp79g" event={"ID":"3c91a73c-eb8e-4396-ad08-2b0f5a9af552","Type":"ContainerDied","Data":"6786e67d027ee3fdad755f2265d1f2e0b989ad5916096295c9bb642dab86877e"} Jul 14 18:14:01.415070 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:01.414746 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4f6333-pp79g" event={"ID":"3c91a73c-eb8e-4396-ad08-2b0f5a9af552","Type":"ContainerStarted","Data":"276533bbf808e175df4e62c2de71508b4f452ea258ec8d82530f3622b6ba7b3b"} Jul 14 18:14:02.551666 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:02.551642 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4f6333-pp79g" Jul 14 18:14:02.566481 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:02.566450 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c91a73c-eb8e-4396-ad08-2b0f5a9af552-maas-api-service-ca\") pod \"3c91a73c-eb8e-4396-ad08-2b0f5a9af552\" (UID: \"3c91a73c-eb8e-4396-ad08-2b0f5a9af552\") " Jul 14 18:14:02.566903 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:02.566873 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c91a73c-eb8e-4396-ad08-2b0f5a9af552-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3c91a73c-eb8e-4396-ad08-2b0f5a9af552" (UID: "3c91a73c-eb8e-4396-ad08-2b0f5a9af552"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 18:14:02.667909 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:02.667867 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c91a73c-eb8e-4396-ad08-2b0f5a9af552-maas-api-service-ca\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:14:03.427761 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:03.427732 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4f6333-pp79g" Jul 14 18:14:03.427761 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:03.427749 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4f6333-pp79g" event={"ID":"3c91a73c-eb8e-4396-ad08-2b0f5a9af552","Type":"ContainerDied","Data":"276533bbf808e175df4e62c2de71508b4f452ea258ec8d82530f3622b6ba7b3b"} Jul 14 18:14:03.427997 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:03.427792 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="276533bbf808e175df4e62c2de71508b4f452ea258ec8d82530f3622b6ba7b3b" Jul 14 18:14:10.471814 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:10.471785 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-4f6333-594b47c87c-7z49f"] Jul 14 18:14:10.472287 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:10.472038 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f6333-594b47c87c-7z49f" podUID="bfb6a4b7-6c11-476d-ae0a-e51f953de711" containerName="maas-api" containerID="cri-o://9fa8a1de0a1d6b646472e1ed6f29f0510159512bc2017743b22f40b3b7e488d6" gracePeriod=30 Jul 14 18:14:10.918476 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:10.918454 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f6333-594b47c87c-7z49f" Jul 14 18:14:11.044471 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:11.044397 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r2dn7\" (UniqueName: \"kubernetes.io/projected/bfb6a4b7-6c11-476d-ae0a-e51f953de711-kube-api-access-r2dn7\") pod \"bfb6a4b7-6c11-476d-ae0a-e51f953de711\" (UID: \"bfb6a4b7-6c11-476d-ae0a-e51f953de711\") " Jul 14 18:14:11.044612 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:11.044480 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bfb6a4b7-6c11-476d-ae0a-e51f953de711-maas-api-tls\") pod \"bfb6a4b7-6c11-476d-ae0a-e51f953de711\" (UID: \"bfb6a4b7-6c11-476d-ae0a-e51f953de711\") " Jul 14 18:14:11.044612 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:11.044512 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfb6a4b7-6c11-476d-ae0a-e51f953de711-openshift-service-ca\") pod \"bfb6a4b7-6c11-476d-ae0a-e51f953de711\" (UID: \"bfb6a4b7-6c11-476d-ae0a-e51f953de711\") " Jul 14 18:14:11.044919 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:11.044893 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfb6a4b7-6c11-476d-ae0a-e51f953de711-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bfb6a4b7-6c11-476d-ae0a-e51f953de711" (UID: "bfb6a4b7-6c11-476d-ae0a-e51f953de711"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 18:14:11.046421 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:11.046392 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bfb6a4b7-6c11-476d-ae0a-e51f953de711-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bfb6a4b7-6c11-476d-ae0a-e51f953de711" (UID: "bfb6a4b7-6c11-476d-ae0a-e51f953de711"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 18:14:11.046518 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:11.046398 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfb6a4b7-6c11-476d-ae0a-e51f953de711-kube-api-access-r2dn7" (OuterVolumeSpecName: "kube-api-access-r2dn7") pod "bfb6a4b7-6c11-476d-ae0a-e51f953de711" (UID: "bfb6a4b7-6c11-476d-ae0a-e51f953de711"). InnerVolumeSpecName "kube-api-access-r2dn7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:14:11.145913 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:11.145890 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bfb6a4b7-6c11-476d-ae0a-e51f953de711-maas-api-tls\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:14:11.146017 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:11.145912 2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfb6a4b7-6c11-476d-ae0a-e51f953de711-openshift-service-ca\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:14:11.146017 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:11.145934 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2dn7\" (UniqueName: \"kubernetes.io/projected/bfb6a4b7-6c11-476d-ae0a-e51f953de711-kube-api-access-r2dn7\") on node \"ip-10-0-132-13.ec2.internal\" DevicePath \"\"" Jul 14 18:14:11.477229 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:11.477180 2554 generic.go:356] "Generic (PLEG): container finished" podID="bfb6a4b7-6c11-476d-ae0a-e51f953de711" containerID="9fa8a1de0a1d6b646472e1ed6f29f0510159512bc2017743b22f40b3b7e488d6" exitCode=0 Jul 14 18:14:11.477618 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:11.477241 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f6333-594b47c87c-7z49f" Jul 14 18:14:11.477618 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:11.477272 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f6333-594b47c87c-7z49f" event={"ID":"bfb6a4b7-6c11-476d-ae0a-e51f953de711","Type":"ContainerDied","Data":"9fa8a1de0a1d6b646472e1ed6f29f0510159512bc2017743b22f40b3b7e488d6"} Jul 14 18:14:11.477618 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:11.477323 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f6333-594b47c87c-7z49f" event={"ID":"bfb6a4b7-6c11-476d-ae0a-e51f953de711","Type":"ContainerDied","Data":"2d631e9a92f6a3adbb3ac87a4442f370df16ddec608137745d7b40910709aee3"} Jul 14 18:14:11.477618 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:11.477344 2554 scope.go:117] "RemoveContainer" containerID="9fa8a1de0a1d6b646472e1ed6f29f0510159512bc2017743b22f40b3b7e488d6" Jul 14 18:14:11.486681 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:11.486664 2554 scope.go:117] "RemoveContainer" containerID="9fa8a1de0a1d6b646472e1ed6f29f0510159512bc2017743b22f40b3b7e488d6" Jul 14 18:14:11.486936 ip-10-0-132-13 kubenswrapper[2554]: E0714 18:14:11.486919 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fa8a1de0a1d6b646472e1ed6f29f0510159512bc2017743b22f40b3b7e488d6\": container with ID starting with 9fa8a1de0a1d6b646472e1ed6f29f0510159512bc2017743b22f40b3b7e488d6 not found: ID does not exist" containerID="9fa8a1de0a1d6b646472e1ed6f29f0510159512bc2017743b22f40b3b7e488d6" Jul 14 18:14:11.486983 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:11.486946 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fa8a1de0a1d6b646472e1ed6f29f0510159512bc2017743b22f40b3b7e488d6"} err="failed to get container status \"9fa8a1de0a1d6b646472e1ed6f29f0510159512bc2017743b22f40b3b7e488d6\": rpc error: code = NotFound desc = could not find container \"9fa8a1de0a1d6b646472e1ed6f29f0510159512bc2017743b22f40b3b7e488d6\": container with ID starting with 9fa8a1de0a1d6b646472e1ed6f29f0510159512bc2017743b22f40b3b7e488d6 not found: ID does not exist" Jul 14 18:14:11.499737 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:11.499709 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-4f6333-594b47c87c-7z49f"] Jul 14 18:14:11.503952 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:11.503929 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-4f6333-594b47c87c-7z49f"] Jul 14 18:14:12.388011 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:12.387979 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bfb6a4b7-6c11-476d-ae0a-e51f953de711" path="/var/lib/kubelet/pods/bfb6a4b7-6c11-476d-ae0a-e51f953de711/volumes" Jul 14 18:14:13.551350 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:13.551315 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-swz27"] Jul 14 18:14:31.851481 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:31.851456 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-87kp7_392297d1-a5b9-4967-9cd6-8d019503f41a/server/0.log" Jul 14 18:14:32.333193 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:32.333112 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f54d68b84-sl8jk_b4e50eb9-ba03-4bcd-82fe-3bf657fab044/copy-manifests/0.log" Jul 14 18:14:32.340323 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:32.340294 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f54d68b84-sl8jk_b4e50eb9-ba03-4bcd-82fe-3bf657fab044/manager/0.log" Jul 14 18:14:32.915457 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:32.915430 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29734200-ss78q_24730953-c360-4cda-a964-4fb4a67485fb/cleanup/0.log" Jul 14 18:14:33.025872 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:33.025845 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-06d94c77-v4pk5_6b65a60c-9c51-4f91-aa0c-dda907bacd0a/revoke-keys/0.log" Jul 14 18:14:33.135253 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:33.135194 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-efd93356-nv84v_8aca8919-29bb-47e5-a86e-454003ad6ccf/revoke-keys/0.log" Jul 14 18:14:33.247161 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:33.247064 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-e8afdc-gtjvs_9d08e2e3-02bf-4809-9ddf-612135c667b5/revoke-keys/0.log" Jul 14 18:14:33.354640 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:33.354615 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-e9d9a7-tx8tg_5e3e98bf-1608-4094-9ce8-5102c9d610e0/revoke-keys/0.log" Jul 14 18:14:33.464178 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:33.464151 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-derive-eef895f6-q89w5_4d358b77-9bcf-4e15-8fff-5ca588992aa7/revoke-keys/0.log" Jul 14 18:14:33.574022 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:33.573943 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-4ef174-szwjv_84789fb0-ff2f-4541-a053-facce71b5200/revoke-keys/0.log" Jul 14 18:14:33.683992 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:33.683967 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-2d5136-qrrkv_f0b7e330-6661-4e9a-9b97-c0a2079c33f9/revoke-keys/0.log" Jul 14 18:14:33.798168 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:33.798137 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-migrate-7d767796-wkgv7_07845d1e-91ff-498a-bf41-78366f429027/revoke-keys/0.log" Jul 14 18:14:33.911858 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:33.911831 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-3d8a768b-9cpfn_206c6626-f5e4-47ab-8451-4f87616cbf9a/revoke-keys/0.log" Jul 14 18:14:34.021964 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:34.021935 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-f92df6e4-nfgxn_b2dd3bc1-0dc8-46b9-905a-f143ae2a975c/revoke-keys/0.log" Jul 14 18:14:34.133171 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:34.133145 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-8bfa74-c4hzk_bbd65c81-140c-49af-b200-ade9e6b5e94d/revoke-keys/0.log" Jul 14 18:14:34.248758 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:34.248683 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-4f6333-pp79g_3c91a73c-eb8e-4396-ad08-2b0f5a9af552/revoke-keys/0.log" Jul 14 18:14:35.129992 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:35.129961 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mvnqq_1f45d5d7-31ab-42d5-a787-c06a1fe43b2f/util/0.log" Jul 14 18:14:35.135857 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:35.135815 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mvnqq_1f45d5d7-31ab-42d5-a787-c06a1fe43b2f/pull/0.log" Jul 14 18:14:35.142009 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:35.141984 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mvnqq_1f45d5d7-31ab-42d5-a787-c06a1fe43b2f/extract/0.log" Jul 14 18:14:35.258898 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:35.258871 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09znmf_1bf1435a-7689-4366-879b-562c985f0d65/util/0.log" Jul 14 18:14:35.265839 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:35.265805 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09znmf_1bf1435a-7689-4366-879b-562c985f0d65/pull/0.log" Jul 14 18:14:35.272249 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:35.272233 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09znmf_1bf1435a-7689-4366-879b-562c985f0d65/extract/0.log" Jul 14 18:14:35.385329 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:35.385266 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wcnzk_0045d523-1b12-4f86-bc45-e490ec45a739/util/0.log" Jul 14 18:14:35.391729 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:35.391709 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wcnzk_0045d523-1b12-4f86-bc45-e490ec45a739/pull/0.log" Jul 14 18:14:35.398293 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:35.398275 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wcnzk_0045d523-1b12-4f86-bc45-e490ec45a739/extract/0.log" Jul 14 18:14:35.512919 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:35.512881 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2rwx_4e9fc54e-180e-44ec-aaa1-76638d1f30ab/extract/0.log" Jul 14 18:14:35.519094 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:35.519078 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2rwx_4e9fc54e-180e-44ec-aaa1-76638d1f30ab/util/0.log" Jul 14 18:14:35.525736 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:35.525712 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2rwx_4e9fc54e-180e-44ec-aaa1-76638d1f30ab/pull/0.log" Jul 14 18:14:36.332508 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:36.332476 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-swz27_ffbc8fa7-8283-4f2b-9e46-724ef9ac1c41/limitador/0.log" Jul 14 18:14:36.924488 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:36.924458 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-p5nfg_5b775922-7477-4d67-91c5-d359eda0ebae/discovery/0.log" Jul 14 18:14:38.200433 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:38.200404 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-pmzkj_c107c170-61cc-46a8-bc76-bef865a6f3b7/main/0.log" Jul 14 18:14:45.066876 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:45.066832 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-88sg9_69371a04-a55d-4548-a7df-2868cd880309/global-pull-secret-syncer/0.log" Jul 14 18:14:45.209937 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:45.209908 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-djms6_210c935e-f5d4-4de0-abee-7d5e2f555f34/konnectivity-agent/0.log" Jul 14 18:14:45.277751 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:45.277718 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-13.ec2.internal_b3ea3cd1054689f0ec1aa0dca0048bb1/haproxy/0.log" Jul 14 18:14:49.837645 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:49.837614 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mvnqq_1f45d5d7-31ab-42d5-a787-c06a1fe43b2f/extract/0.log" Jul 14 18:14:49.861568 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:49.861537 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mvnqq_1f45d5d7-31ab-42d5-a787-c06a1fe43b2f/util/0.log" Jul 14 18:14:49.888085 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:49.888057 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mvnqq_1f45d5d7-31ab-42d5-a787-c06a1fe43b2f/pull/0.log" Jul 14 18:14:49.921219 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:49.921194 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09znmf_1bf1435a-7689-4366-879b-562c985f0d65/extract/0.log" Jul 14 18:14:49.945508 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:49.945481 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09znmf_1bf1435a-7689-4366-879b-562c985f0d65/util/0.log" Jul 14 18:14:49.967888 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:49.967863 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09znmf_1bf1435a-7689-4366-879b-562c985f0d65/pull/0.log" Jul 14 18:14:50.006463 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:50.006433 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wcnzk_0045d523-1b12-4f86-bc45-e490ec45a739/extract/0.log" Jul 14 18:14:50.049474 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:50.049446 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wcnzk_0045d523-1b12-4f86-bc45-e490ec45a739/util/0.log" Jul 14 18:14:50.074704 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:50.074679 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wcnzk_0045d523-1b12-4f86-bc45-e490ec45a739/pull/0.log" Jul 14 18:14:50.108361 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:50.108335 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2rwx_4e9fc54e-180e-44ec-aaa1-76638d1f30ab/extract/0.log" Jul 14 18:14:50.132265 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:50.132241 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2rwx_4e9fc54e-180e-44ec-aaa1-76638d1f30ab/util/0.log" Jul 14 18:14:50.153361 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:50.153326 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2rwx_4e9fc54e-180e-44ec-aaa1-76638d1f30ab/pull/0.log" Jul 14 18:14:51.003048 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:51.003022 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-swz27_ffbc8fa7-8283-4f2b-9e46-724ef9ac1c41/limitador/0.log" Jul 14 18:14:52.760835 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:52.760812 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_14bf1b4f-bd69-4e71-af6e-8eb73d1c7087/alertmanager/0.log" Jul 14 18:14:52.785331 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:52.785304 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_14bf1b4f-bd69-4e71-af6e-8eb73d1c7087/config-reloader/0.log" Jul 14 18:14:52.812277 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:52.812258 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_14bf1b4f-bd69-4e71-af6e-8eb73d1c7087/kube-rbac-proxy-web/0.log" Jul 14 18:14:52.838290 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:52.838274 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_14bf1b4f-bd69-4e71-af6e-8eb73d1c7087/kube-rbac-proxy/0.log" Jul 14 18:14:52.867232 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:52.867182 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_14bf1b4f-bd69-4e71-af6e-8eb73d1c7087/kube-rbac-proxy-metric/0.log" Jul 14 18:14:52.899814 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:52.899789 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_14bf1b4f-bd69-4e71-af6e-8eb73d1c7087/prom-label-proxy/0.log" Jul 14 18:14:52.931908 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:52.931889 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_14bf1b4f-bd69-4e71-af6e-8eb73d1c7087/init-config-reloader/0.log" Jul 14 18:14:53.227500 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:53.227473 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lglc8_798ec85b-c0a7-4738-a711-d72c0c6c8b4a/node-exporter/0.log" Jul 14 18:14:53.248175 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:53.248152 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lglc8_798ec85b-c0a7-4738-a711-d72c0c6c8b4a/kube-rbac-proxy/0.log" Jul 14 18:14:53.270515 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:53.270492 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lglc8_798ec85b-c0a7-4738-a711-d72c0c6c8b4a/init-textfile/0.log" Jul 14 18:14:53.646924 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:53.646896 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-8hl55_da755c27-b962-4855-8616-aa05a3bdab0f/prometheus-operator/0.log" Jul 14 18:14:53.665997 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:53.665970 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-8hl55_da755c27-b962-4855-8616-aa05a3bdab0f/kube-rbac-proxy/0.log" Jul 14 18:14:53.689162 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:53.689143 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-hlw49_1dd534d0-8b90-42c4-8d00-8e11079fb474/prometheus-operator-admission-webhook/0.log" Jul 14 18:14:53.787032 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:53.787008 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-777fb8f587-jwff6_ab2cd57e-c525-4003-99b4-488ca8f02e9e/thanos-query/0.log" Jul 14 18:14:53.809719 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:53.809685 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-777fb8f587-jwff6_ab2cd57e-c525-4003-99b4-488ca8f02e9e/kube-rbac-proxy-web/0.log" Jul 14 18:14:53.831622 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:53.831592 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-777fb8f587-jwff6_ab2cd57e-c525-4003-99b4-488ca8f02e9e/kube-rbac-proxy/0.log" Jul 14 18:14:53.851951 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:53.851931 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-777fb8f587-jwff6_ab2cd57e-c525-4003-99b4-488ca8f02e9e/prom-label-proxy/0.log" Jul 14 18:14:53.872360 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:53.872342 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-777fb8f587-jwff6_ab2cd57e-c525-4003-99b4-488ca8f02e9e/kube-rbac-proxy-rules/0.log" Jul 14 18:14:53.893465 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:53.893436 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-777fb8f587-jwff6_ab2cd57e-c525-4003-99b4-488ca8f02e9e/kube-rbac-proxy-metrics/0.log" Jul 14 18:14:54.985191 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:54.985155 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4c4r8/perf-node-gather-daemonset-k276x"] Jul 14 18:14:54.985574 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:54.985519 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c91a73c-eb8e-4396-ad08-2b0f5a9af552" containerName="revoke-keys" Jul 14 18:14:54.985574 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:54.985532 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c91a73c-eb8e-4396-ad08-2b0f5a9af552" containerName="revoke-keys" Jul 14 18:14:54.985574 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:54.985547 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfb6a4b7-6c11-476d-ae0a-e51f953de711" containerName="maas-api" Jul 14 18:14:54.985574 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:54.985553 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfb6a4b7-6c11-476d-ae0a-e51f953de711" containerName="maas-api" Jul 14 18:14:54.985709 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:54.985614 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c91a73c-eb8e-4396-ad08-2b0f5a9af552" containerName="revoke-keys" Jul 14 18:14:54.985709 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:54.985626 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfb6a4b7-6c11-476d-ae0a-e51f953de711" containerName="maas-api" Jul 14 18:14:54.988687 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:54.988671 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4c4r8/perf-node-gather-daemonset-k276x" Jul 14 18:14:54.991177 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:54.991157 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4c4r8\"/\"kube-root-ca.crt\"" Jul 14 18:14:54.992439 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:54.992420 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4c4r8\"/\"default-dockercfg-vpwkq\"" Jul 14 18:14:54.992533 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:54.992431 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4c4r8\"/\"openshift-service-ca.crt\"" Jul 14 18:14:54.998250 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:54.998228 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4c4r8/perf-node-gather-daemonset-k276x"] Jul 14 18:14:55.021922 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:55.021897 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/806ea858-1781-4ba0-bb16-69b8fe96fac8-podres\") pod \"perf-node-gather-daemonset-k276x\" (UID: \"806ea858-1781-4ba0-bb16-69b8fe96fac8\") " pod="openshift-must-gather-4c4r8/perf-node-gather-daemonset-k276x" Jul 14 18:14:55.022027 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:55.021934 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/806ea858-1781-4ba0-bb16-69b8fe96fac8-lib-modules\") pod \"perf-node-gather-daemonset-k276x\" (UID: \"806ea858-1781-4ba0-bb16-69b8fe96fac8\") " pod="openshift-must-gather-4c4r8/perf-node-gather-daemonset-k276x" Jul 14 18:14:55.022027 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:55.021968 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/806ea858-1781-4ba0-bb16-69b8fe96fac8-sys\") pod \"perf-node-gather-daemonset-k276x\" (UID: \"806ea858-1781-4ba0-bb16-69b8fe96fac8\") " pod="openshift-must-gather-4c4r8/perf-node-gather-daemonset-k276x" Jul 14 18:14:55.022101 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:55.022033 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kmhv\" (UniqueName: \"kubernetes.io/projected/806ea858-1781-4ba0-bb16-69b8fe96fac8-kube-api-access-9kmhv\") pod \"perf-node-gather-daemonset-k276x\" (UID: \"806ea858-1781-4ba0-bb16-69b8fe96fac8\") " pod="openshift-must-gather-4c4r8/perf-node-gather-daemonset-k276x" Jul 14 18:14:55.022137 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:55.022096 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/806ea858-1781-4ba0-bb16-69b8fe96fac8-proc\") pod \"perf-node-gather-daemonset-k276x\" (UID: \"806ea858-1781-4ba0-bb16-69b8fe96fac8\") " pod="openshift-must-gather-4c4r8/perf-node-gather-daemonset-k276x" Jul 14 18:14:55.122937 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:55.122902 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/806ea858-1781-4ba0-bb16-69b8fe96fac8-proc\") pod \"perf-node-gather-daemonset-k276x\" (UID: \"806ea858-1781-4ba0-bb16-69b8fe96fac8\") " pod="openshift-must-gather-4c4r8/perf-node-gather-daemonset-k276x" Jul 14 18:14:55.123093 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:55.122950 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/806ea858-1781-4ba0-bb16-69b8fe96fac8-podres\") pod \"perf-node-gather-daemonset-k276x\" (UID: \"806ea858-1781-4ba0-bb16-69b8fe96fac8\") " pod="openshift-must-gather-4c4r8/perf-node-gather-daemonset-k276x" Jul 14 18:14:55.123093 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:55.123031 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/806ea858-1781-4ba0-bb16-69b8fe96fac8-proc\") pod \"perf-node-gather-daemonset-k276x\" (UID: \"806ea858-1781-4ba0-bb16-69b8fe96fac8\") " pod="openshift-must-gather-4c4r8/perf-node-gather-daemonset-k276x" Jul 14 18:14:55.123093 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:55.123061 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/806ea858-1781-4ba0-bb16-69b8fe96fac8-podres\") pod \"perf-node-gather-daemonset-k276x\" (UID: \"806ea858-1781-4ba0-bb16-69b8fe96fac8\") " pod="openshift-must-gather-4c4r8/perf-node-gather-daemonset-k276x" Jul 14 18:14:55.123093 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:55.123065 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/806ea858-1781-4ba0-bb16-69b8fe96fac8-lib-modules\") pod \"perf-node-gather-daemonset-k276x\" (UID: \"806ea858-1781-4ba0-bb16-69b8fe96fac8\") " pod="openshift-must-gather-4c4r8/perf-node-gather-daemonset-k276x" Jul 14 18:14:55.123257 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:55.123111 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/806ea858-1781-4ba0-bb16-69b8fe96fac8-sys\") pod \"perf-node-gather-daemonset-k276x\" (UID: \"806ea858-1781-4ba0-bb16-69b8fe96fac8\") " pod="openshift-must-gather-4c4r8/perf-node-gather-daemonset-k276x" Jul 14 18:14:55.123257 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:55.123139 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kmhv\" (UniqueName: \"kubernetes.io/projected/806ea858-1781-4ba0-bb16-69b8fe96fac8-kube-api-access-9kmhv\") pod \"perf-node-gather-daemonset-k276x\" (UID: \"806ea858-1781-4ba0-bb16-69b8fe96fac8\") " pod="openshift-must-gather-4c4r8/perf-node-gather-daemonset-k276x" Jul 14 18:14:55.123257 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:55.123200 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/806ea858-1781-4ba0-bb16-69b8fe96fac8-lib-modules\") pod \"perf-node-gather-daemonset-k276x\" (UID: \"806ea858-1781-4ba0-bb16-69b8fe96fac8\") " pod="openshift-must-gather-4c4r8/perf-node-gather-daemonset-k276x" Jul 14 18:14:55.123257 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:55.123250 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/806ea858-1781-4ba0-bb16-69b8fe96fac8-sys\") pod \"perf-node-gather-daemonset-k276x\" (UID: \"806ea858-1781-4ba0-bb16-69b8fe96fac8\") " pod="openshift-must-gather-4c4r8/perf-node-gather-daemonset-k276x" Jul 14 18:14:55.131556 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:55.131533 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kmhv\" (UniqueName: \"kubernetes.io/projected/806ea858-1781-4ba0-bb16-69b8fe96fac8-kube-api-access-9kmhv\") pod \"perf-node-gather-daemonset-k276x\" (UID: \"806ea858-1781-4ba0-bb16-69b8fe96fac8\") " pod="openshift-must-gather-4c4r8/perf-node-gather-daemonset-k276x" Jul 14 18:14:55.299436 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:55.299351 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4c4r8/perf-node-gather-daemonset-k276x" Jul 14 18:14:55.428493 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:55.428465 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4c4r8/perf-node-gather-daemonset-k276x"] Jul 14 18:14:55.429495 ip-10-0-132-13 kubenswrapper[2554]: W0714 18:14:55.429472 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod806ea858_1781_4ba0_bb16_69b8fe96fac8.slice/crio-ae6229fa1e0fb7133dae7c3a14a5a815b3c80346823349ed4e8f3b9e01eeedd0 WatchSource:0}: Error finding container ae6229fa1e0fb7133dae7c3a14a5a815b3c80346823349ed4e8f3b9e01eeedd0: Status 404 returned error can't find the container with id ae6229fa1e0fb7133dae7c3a14a5a815b3c80346823349ed4e8f3b9e01eeedd0 Jul 14 18:14:55.721808 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:55.721771 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4c4r8/perf-node-gather-daemonset-k276x" event={"ID":"806ea858-1781-4ba0-bb16-69b8fe96fac8","Type":"ContainerStarted","Data":"71acde085be5e73a350712657276673f25032e0773236bb5cfe73496a90238c0"} Jul 14 18:14:55.721998 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:55.721818 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4c4r8/perf-node-gather-daemonset-k276x" event={"ID":"806ea858-1781-4ba0-bb16-69b8fe96fac8","Type":"ContainerStarted","Data":"ae6229fa1e0fb7133dae7c3a14a5a815b3c80346823349ed4e8f3b9e01eeedd0"} Jul 14 18:14:55.721998 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:55.721877 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4c4r8/perf-node-gather-daemonset-k276x" Jul 14 18:14:55.741232 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:55.741171 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4c4r8/perf-node-gather-daemonset-k276x" podStartSLOduration=1.741158849 podStartE2EDuration="1.741158849s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 18:14:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:14:55.738405997 +0000 UTC m=+2939.879654703" watchObservedRunningTime="2026-07-14 18:14:55.741158849 +0000 UTC m=+2939.882407553" Jul 14 18:14:56.197130 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:56.197104 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5b6ff48bc7-dss8d_7c9401b9-de05-45f2-84fa-882ffbee6e53/console/0.log" Jul 14 18:14:56.230990 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:56.230960 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-vgj5f_9415d694-c5e5-492d-a2f2-348323c40cd7/download-server/0.log" Jul 14 18:14:57.878698 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:57.878672 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hbrtv_cb86b8d6-ce99-4d8a-8170-2c421c00208a/dns/0.log" Jul 14 18:14:57.901518 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:57.901495 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hbrtv_cb86b8d6-ce99-4d8a-8170-2c421c00208a/kube-rbac-proxy/0.log" Jul 14 18:14:57.999507 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:57.999477 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-b8gnk_7768649b-c82e-4a04-a4c1-93061292bcec/dns-node-resolver/0.log" Jul 14 18:14:58.606725 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:58.606699 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7ffq4_7f7817bc-7352-4f6a-9f20-cf101d130a7c/node-ca/0.log" Jul 14 18:14:59.662272 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:14:59.662243 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-p5nfg_5b775922-7477-4d67-91c5-d359eda0ebae/discovery/0.log" Jul 14 18:15:00.143011 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:15:00.142975 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734215-42zdp"] Jul 14 18:15:00.147538 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:15:00.147523 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734215-42zdp" Jul 14 18:15:00.150593 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:15:00.150573 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9rnpk\"" Jul 14 18:15:00.152162 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:15:00.152133 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734215-42zdp"] Jul 14 18:15:00.160024 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:15:00.160008 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734215-42zdp" Jul 14 18:15:00.287596 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:15:00.287566 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734215-42zdp"] Jul 14 18:15:00.287979 ip-10-0-132-13 kubenswrapper[2554]: W0714 18:15:00.287953 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc75eaf2_8133_4bea_be87_fb769b216309.slice/crio-f41aeb83367a5971855c4aa5000745087649603dc8d73ca2d64bfe12062f032e WatchSource:0}: Error finding container f41aeb83367a5971855c4aa5000745087649603dc8d73ca2d64bfe12062f032e: Status 404 returned error can't find the container with id f41aeb83367a5971855c4aa5000745087649603dc8d73ca2d64bfe12062f032e Jul 14 18:15:00.593794 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:15:00.593770 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bwrcq_8e360557-5281-4fd8-b27a-7ec139ccc92b/serve-healthcheck-canary/0.log" Jul 14 18:15:00.752683 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:15:00.752591 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734215-42zdp" event={"ID":"cc75eaf2-8133-4bea-be87-fb769b216309","Type":"ContainerStarted","Data":"dbd657b0565f1364efed596344b625989103dea59d3265ab75c1867c07cf896b"} Jul 14 18:15:00.752683 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:15:00.752656 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734215-42zdp" event={"ID":"cc75eaf2-8133-4bea-be87-fb769b216309","Type":"ContainerStarted","Data":"f41aeb83367a5971855c4aa5000745087649603dc8d73ca2d64bfe12062f032e"} Jul 14 18:15:00.767793 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:15:00.767742 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734215-42zdp" podStartSLOduration=0.767723442 podStartE2EDuration="767.723442ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:15:00.767177516 +0000 UTC m=+2944.908426220" watchObservedRunningTime="2026-07-14 18:15:00.767723442 +0000 UTC m=+2944.908972146" Jul 14 18:15:01.272925 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:15:01.272901 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lzjdm_b038901e-a28d-4de8-a035-194c93348967/kube-rbac-proxy/0.log" Jul 14 18:15:01.292716 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:15:01.292690 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lzjdm_b038901e-a28d-4de8-a035-194c93348967/exporter/0.log" Jul 14 18:15:01.314083 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:15:01.314053 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lzjdm_b038901e-a28d-4de8-a035-194c93348967/extractor/0.log" Jul 14 18:15:01.738573 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:15:01.738550 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4c4r8/perf-node-gather-daemonset-k276x" Jul 14 18:15:01.760847 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:15:01.760815 2554 generic.go:356] "Generic (PLEG): container finished" podID="cc75eaf2-8133-4bea-be87-fb769b216309" containerID="dbd657b0565f1364efed596344b625989103dea59d3265ab75c1867c07cf896b" exitCode=0 Jul 14 18:15:01.761233 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:15:01.760857 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734215-42zdp" event={"ID":"cc75eaf2-8133-4bea-be87-fb769b216309","Type":"ContainerDied","Data":"dbd657b0565f1364efed596344b625989103dea59d3265ab75c1867c07cf896b"} Jul 14 18:15:02.897448 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:15:02.897423 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734215-42zdp" Jul 14 18:15:03.773544 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:15:03.773518 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734215-42zdp" Jul 14 18:15:03.773733 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:15:03.773553 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734215-42zdp" event={"ID":"cc75eaf2-8133-4bea-be87-fb769b216309","Type":"ContainerDied","Data":"f41aeb83367a5971855c4aa5000745087649603dc8d73ca2d64bfe12062f032e"} Jul 14 18:15:03.773733 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:15:03.773608 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f41aeb83367a5971855c4aa5000745087649603dc8d73ca2d64bfe12062f032e" Jul 14 18:15:03.865104 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:15:03.865081 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-87kp7_392297d1-a5b9-4967-9cd6-8d019503f41a/server/0.log" Jul 14 18:15:04.192384 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:15:04.192283 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f54d68b84-sl8jk_b4e50eb9-ba03-4bcd-82fe-3bf657fab044/manager/0.log" Jul 14 18:15:04.218857 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:15:04.218830 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f54d68b84-sl8jk_b4e50eb9-ba03-4bcd-82fe-3bf657fab044/copy-manifests/0.log" Jul 14 18:15:05.564053 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:15:05.564020 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7cb94d9664-482xx_11bc76d0-ed17-4b7f-b7e3-b8d2458b136e/manager/0.log" Jul 14 18:15:05.615074 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:15:05.615039 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-mxphq_b05692f3-24b2-4b79-b964-7dfb9adfa5a0/openshift-lws-operator/0.log" Jul 14 18:15:06.019737 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:15:06.019701 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-p5nfg_5b775922-7477-4d67-91c5-d359eda0ebae/discovery/0.log" Jul 14 18:15:07.450324 ip-10-0-132-13 kubenswrapper[2554]: I0714 18:15:07.450284 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-p5nfg_5b775922-7477-4d67-91c5-d359eda0ebae/discovery/0.log"